I've tried not using the arrow keys, but my problem is that I use the dvorak layout at home, and typing jcvp to maneuver around (what hjkl maps to) confuses me too much. Especially since I use vim at work as well on the normal qwerty layout. Maybe I'll try again soon. In the meantime, I'll try to incorporate *, #, and gi into my muscle memory (for both layouts).

The dvorak layout (and especially switching layouts) saved me from a lot of pain, so I don't plan on ditching it any time soon.