Because that wouldn't work for copying and pasting code nor writing shell scripts. If you're going down that route then really you need a printable character to prefix the command line (or similar solution)
For what it's worth though, most shells (including my own one) do support switching between different text entry / hotkey modes - such as vi - even if the syntax is consistent.
Maybe something based on Color Forth's ideas?