Same solution as you, I use different editors in different contexts. You use Emacs and Vim, I use Sublime and Vim. I also edit locally and push my git branch to the remote server to test/run the code. I do that for most bigger changes and use vim to do smaller debugging changes.

Some people use plugins to edit code over a tunnel using sftp. One of my coworkers edits locally and has aliases to rsync and run his code on the remote machine.