Making it fast while still using it, not sure. But I can share an alternative, since a friend had the same issue and this was a literally ten times faster for a lot of small files (in the order of 30 minutes instead of 5 hours):
cd path/to/target/location
ssh user@target 'tar c /tmp/example' | tar x
Quick guide to tar, since it's super simple: c for compress
x for extract
f for file (since we send it to stdout / read from stdin, I don't use the f option)
v for verbosity (not sure if that works on the remote side)
z for zlib compression (ssh can do compression already, so also unused here)
t for testing (reading) an archive without extracting it ("tar tv <your.tar" will show you the contents, it's almost like real TV!)
That's all I've ever needed.So what this will do is run "tar create <directory>" on the remote system and, tar being a classic tool, it'll just output that binary archive data to stdout since you didn't specify a file ("tar cf your.tar <directory>"). On the receiving side, you pipe it to another tar command that reads from stdin (since, again, no file was specified) and extracts the files from the incoming data stream.
At the moment I use intellij which has a "sync with remote" function; I can run the things local and push them to the remote testing environment whenever I feel like it.
(my predecessor would use git a lot, ugh)
rsync -av lcldir/ x:remotedir/
rsync -av x:remotedir/ lcldir/
I've gotten pretty strict about trailing slashes otherwise I end up creating directories within directories depending on whether things exist or not
Still looking for a way to make sftp/scp work fast.