1. On macOS, host FS volumes are two magnitudes slower. There are a series of hacks that try to mitigate this problem and progress has been made, but it’s still too slow for a dev environment. Linux doesn’t have this problem. I don’t know about Windows.
2. Also on macOS, the host networking bridge doesn’t work the same way it works on Linux. There’s a different hostname that’s used to access the host from within the container that’s inconsistent with Docker on Linux.
Docker is working through these issues, but they’ve been ongoing for years.
I’ve written about Docker dev macOS environments at bradgessler.com, but have given up mostly because FS performance is so poor. Now I spin up what I call “Dockerish” dev environments where my service dependencies run in Docker and the app runs on my host.
As for Windows, it doesn't have the same fs performance issues on volume mounts, but AFAIK file system events have never worked there [2].
My current reluctant compromise to make volume mounts work seamlessly across platforms is to use polling instead of relying on file system events. That way I can turn on caching on Docker for Mac, and everything just works. The downside is of course polling is inefficient, but I've found when you take care to exclude all non-source directories from the polling, it's usually not as bad as you'd think.
[1] https://docs.docker.com/docker-for-mac/osxfs-caching/#tuning...
[2] https://github.com/docker/for-win/issues/56#issuecomment-242...
It's pretty easy these days to use tools like Vagrant to spin up the VM then run a Docker Compose file or whatever to get your development environment ready, and would result in identical cross platform environments.
I also like that my underlying code is also within an encrypted VM disk, not laying around on my host mac os.
When I use Reason I'll sometimes mount over SSHFS to edit using neovim in visual studio, though right now I'm building native iOS apps with it so it's actually not in my VM so that xcode can build faster
FTFY
There's this Vagrant extension [0] that works pretty well on mac at least (which I actually forked and pushed to rubygems a few years ago when I discovered it because it had been abandoned and no longer worked)
Unfortunately the Vagrant team seems to have no interest in having all 3 of the properties I mentioned just work out of the box, I'm not sure why. Maybe it's just a lack of resources at Hashicorp, I'm not sure how much dev time they put into Vagrant these days since it's otherwise very mature and they have such a large number of open source projects.
But it seems kind of obvious that the functionality needed to get this stuff to work, is an entire network filesystem stack. It seems the obvious choices would be to build into VirtualBox either nfs, or webdav - or leverage host/external support for nfs, cifs, or something like OpenAFS. With sshfs as another pragmatic option. Maybe in the future ipfs will be a viable option too.
But I think cifs is still the "mostly works, with/without complex auth, cross platform" - with webdav a distant second.
I used your vagrant-unison2 plugin for developer environments at Airbnb for a time, although we've now moved on to a more complex, home-grown Unison wrapper with Watchman for filesystem watches.
I set up a TCP server, one light process watches file changes and the other side reacts to it. (Rails). It was one of my silent victories that the entire company used, but I had to do because I was the guinnea pig on the vagrant migration.
On Windows you can register a handler/callback, is that available in Linux?
e.g. tcp mounts, getting read/write blocks matched up btw/client server and sized to be digestable but big enough to move data, etc.
also, nfs is mainly only suited for 'NAS-like' operations - things like rdbms's do waay better on iscsi or eating the vdisk performance.
last I messed with macos nfsd (which has been a while), it a way happier with smaller blocksizes (e.g. 8-64k range) - modern linuces will attempt 1MB which is too much for the older 4.4BSD based code
another thing to look at is timeouts / backoffs - it's easy to kill performance by setting these things too agressively so that the system double-chokes when it gets bogged down..
I did have to supply an extra "mount_options:['actimeo']=1" option to the synced_folder command to make file watcher-type systems work tolerably. With the defaults I'd often be waiting several seconds before any changes made by the host would be picked up by the guest.
Unlike rsync, ZFS always knows what's changed since the last snapshot, so there's no directory scanning.
Unlike inotify, there is (seemingly) no(?) risk of dropped events, which I've experienced with tools like watchman. My ZFS solution still needs prompting event to start the snapshot, but it no longer needs to be told _what_ changed.
Critically the sync flows in one direction. This is a deal breaker for applications that need to communicate file changes back out to the host, but that hasn't been an issue in my use cases.
Unlike NFS, it also means we must (and can) relax the strong consistency model, and so using the filesystem is not so latency bound. This becomes important if the virtual machine is not running locally..
My advice is to cough up an hour or two salary and pay for parallels or VMware (if on Mac)
Vbox (or the Vagrant use of it) is hard coded to use a NAT as the default interface
The common "Vagrant synced folders are slow" relates exclusively to Virtualbox shared folders.
Then there’s the periodic refusal to talk on the network even if the virtual switches are configured properly.
Then the inexplicable “slow boot from hell” which happens randomly where it’ll just hang starting up the kernel at 0% CPU for up to 8 minutes.
This is with CentOS 7 on a gen2 machine.
Honestly though, VMWare is pretty cheap. VMWare Workstation Player and VMWare Fusion aren't cheap upfront, but the upgrade pricing to stay current is decent - I spend more on my JetBrains licenses every year. It's worth it to have a fast and mostly headache-free virtualization experience.
My experience with Vagrant is that it's amazing, but I've only used it for a web application with Github and Jenkins for CI. Dev team doesn't have to spend a second thinking outside the git repo.
1. How to properly do cross-platform, high-performance two-way synced folders between host and guest. Most providers only support one-way syncing. Virtualbox has shared folders, but their performance is pretty lousy and they have issues with relative symlinks. In fact, I still don't fully know what the correct setup is for a dev environment where the files are edited on the host and the guest immediately picks up on them...
2. This idea that with vagrant you'll never again say "It works on my machine" is a lie. So many inconsistencies between Vagrant on Windows, Linux and macOS. Internet connection sharing, line endings issues, symlinking issues, ...
If anyone wants to see how we're using Vagrant:
https://github.com/hearthsim/hearthsim-vagrant
I don't want to make it sound like Vagrant isn't solving a real problem though, it is. It's just not the unicorn it claims to be.