What command line are you talking about? If you want a command-line IPython-style REPL, enter MathKernel, which absolutely does support arrow keys to go backward and forward in history (and works over SSH without X). If you want a IPython (later Jupyter) notebook-style interface, enter Mathematica/Wolfram notebooks (guess where IPython notebook got its idea from). Sounds like you just didn’t bother to learn a bit about it before making up your mind.