This was a revelation for me as well. I also didn't realize that if you haven't configured vi keybindings, the default is Emacs (in bash or anything using readline). Even though Vim's my main editor, I found modal editing a bit too heavyweight on the commandline, so I prefer the default Emacs (most useful by far: C-b to go back one word and C-k to delete everything right of cursor).