...then you're doing it wrong.
If Vi was made in XXI century then we would move the cursor with WASD keys.
I totally agree to this article: if you decided to try Vim — don't punish yourself for that (by disabling arrow keys and mouse)!
Instead, take it easy: learn the basics (modes) and start with simple editing. It's XXI century, modern versions of Gvim have pretty nice system integration for a reason — to help you stop "fighting" your new editor. Vim is not about doing things "the only true way", with minimum keystrokes — it's about doing things efficiently, and if it's easier for you to type xxx than 3x — feel free to do it!
To start loving Vim, try to find out how it can help you: if you do something with too many keystrokes — it probably can be optimized. If you have colleagues using Vim — ask them what useful tricks they know. If you have free time — read articles like this — there you might find useful Vim tricks you'll love Vim for. Print some cheatsheet (like this one) and look through Vim help — for same reason. If you want to do something, but don't know how — ask Google — chances are high there's a built-in feature or plugin for that.
And one day, in some other editor, you will feel uncomfortable and will want to get back to Vim.
For me such "killer feature" was ci" to change text inside "quotes". Later I started using f and t to find and get to specific symbols on current line and understood benefits of multiple registers (clipboards) and macro recording.