With VMware Workstation that feature is super fast. The product has a much better experience overall.
Unfortunately, I didn't have much luck with Vagrant's VMware plugin (paid) and couldn't realistic make the switch.
It seems using each OS's native virtualization (Hyper-V, xhyve, KVM) is the way to go, but that means we have to maintain separate vagrant boxes, which we don't have to with VMware/VirtualBox since they work on Linux/OSX/Windows.
You can of course install whatever remote server you want inside the VM but aside from that you can connect to a VM directly using VRDP.
But you can also manager your entire virtualbox instance using RemoteBox ( http://remotebox.knobgoblin.org.uk/about.cgi ) or from a web GUI using phpVirtualBox ( https://sourceforge.net/p/phpvirtualbox/wiki/Home/ ).
I'm sure there are other options as well.
I have played with both of the alternate solutions you have shared. These are not perfect, unlike what VMware Workstation provides (as it has, for long period of time).
Why do people use VMware workstation over virtualbox?