This was a revelation for me as well. I also didn't realize that if you haven't configured vi keybindings, the default is Emacs (in bash or anything using readline). Even though Vim's my main editor, I found modal editing a bit too heavyweight on the commandline, so I prefer the default Emacs (most useful by far: C-b to go back one word and C-k to delete everything right of cursor).

I prefer the emacs bindings for the command line such as C-A, C-U (mostly due to muscle memory), but have set up my $EDITOR as vim. This allows me to do C-X C-E, which opens the current command in vim to be edited.

If you are using zsh, you need to add this to ur .zshrc

  autoload -z edit-command-line
  zle -N edit-command-line
  bindkey "^X^E" edit-command-line

you can do that in bash too, and any cli that uses readline library

  set -o vi

Starts you in insert mode. To go back to the usual command-line editing,

  set -o emacs

also zsh (tho it doesn't use readline per se)