I use it regularly with VMs. I find it a much better experience to have a bunch of terminals open on my host that are ssh'ed into the VM and can open new GUI windows on the host (generally meld for either git-merge or git-diff).

The main draw is just that it works so seamlessly. All you have to do is remember to add the "-X" flag when you call ssh and everything just works.