Is there too much lag through an SSH connection for you?

I often do things this way. SSH from whatever I'm holding to a more powerful machine with my tools and configs. Then the terminal is on your host, at least.

I've also RDPed into local Windows VMs to get a better experience like working copy/paste. That's the approach Hyper-V took with their enhanced session mode