I generally use VLC by default, but I've found that it frequently stutters in decoding files mounted via sshfs, particularly when I use the scrub bar. I'm guessing that's because sshfs introduces some latency on seeks that upsets the decoders, but other video players don't have the same problem.

So, just anecdotally, VLC isn't perfect, but I'd agree that it's generally quite close.