In that situation I'd likely as not just backspace to the offending location and continue typing. Although I have been known to map caps+hjkl to the arrow keys.
(I'm assuming your situation is a typo while you're inserting a short string, rather than just defaulting to insert mode)
With arrows: <LEFT><LEFT>x<RIGHT>
Without arrows: <ESC>hix<ESC>lli
It also much depends on a keyboard design, the laptop-like often have arrows under ENTER and using them doesn't require hand movement.