If you found this unusable, you probably haven't had to work with the Virtualbox-backed version of Docker for Mac :-)

Have you considered storing the npm_modules and running webpack on a tmpfs/volume (so that it doesn't have to go through the shared FS layers) and only copying the end product to a shared volume?

Or maybe we can realize that npm and webpack are both trash. And we should stop embracing them and move web dev away from this mess.

Don't get me wrong. They were good workaround years ago. But I hope we can do better.

Creating and mounting a loop-volume within the HGFS and then running NPM in there, would probably also be helpful. It certainly is when wrangling a directory with lots of tiny inodes over SMB. (It’s also what Apple themselves do for running Time Machine over SMB: they create a sparsebundle image on the remote and mount it locally, and write to that.)

(The efficient thing about this class of solutions, if you’re wondering, is that the client mounting the loop-image ends up owning + managing the loop-image’s internal filesystem’s metadata within its own local disk cache, such that it can coalesce filesystem metadata writes and only push a new copy of the disk-image blocks backing the filesystem metadata after a potentially-huge number of changes. With a network/host-guest filesystem, meanwhile, every filesystem metadata change must become its own synchronous message to the host, to be pushed to the host’s filesystem driver for linearization, so it can succeed or fail relative to other things going on within the host.)

I used to run my own Xhyve based VM just to avoid the brutal VirtualBox Docker system on macOS. Man that’s some bad memories.

The newer Docker for Mac is better, but file system perf still could be massively improved. Big node_module directories can still cause pain even today

> Big node_module directories can still cause pain

is there any other kind?

Bigger node_modules directories

Yep, using Webpack in Docker on a Mac is essentially unusable. I see around a 5-10x slowdown in Webpack builds. Using docker-sync seems to help the CPU out a bit, so my MacBook Pro doesn't do the jet engine space heater routine quite as much, but the performance is still nearly unusable.

https://docker-sync.readthedocs.io/en/latest/

webpack can be made use-able relatively easily if you do not mount node modules over the shared file system. I've been doing this for quite awhile with a volumes declaration in docker-compose that looks like this (running nextjs, assuming /usr/src/app is where your dockerfile has your node stuff):

(on service definition): volumes: - .:/usr/src/app:cached - node_modules:/usr/src/app/node_modules/ - next_artifacts:/usr/src/app/.next/

and then in the top level volumes key defining node_modules and next_artifacts as blank/default.

That means I mount everything except node modules and the build artifacts so the shared filesystem does a LOT less work trying to sync stuff. The downside, of course, is that I need to run npm commands both inside the container and outside if i want them in my IDE. A fair trade for decent performance. That setup is still not as fast as native but definitely usable and does not send my machine into space header mode much more than normal usage.

Interesting. I have always had the same docker-compose setup with a separate node_modules volume inside Docker, but I've always still had the jet engine space heater issue.

I've been able to improve performance in docker-sync by adding high-churn folders that I don't care about to sync_excludes

Virtualbox-backed Docker on Windows was my first introduction to Docker - it literally put me off touching Docker for 1-2 years!

After eventually falling in love with it on Windows and Linux, I later tried it on MacOS... oh my. Not fun!

The fact you need to partition off RAM and CPUs for Docker on the Mac is the killer for me.

Don’t you have to do that under windows too given WSL2 also runs as a VM?

no not in wsl2. (you can, but there is no need for that) (btw. disk storage can be too small, but that is the only problem that can occur https://docs.microsoft.com/de-de/windows/wsl/compare-version...)

Actually that's done for you, whether you control it or not. WSL2 is essentially a VM with better host integration.

actually under hyper-v windows is a vm aswell, it's just not visible. it's the reason why the hyper-v hypervisor platform was needed.

The shared filesystems are also quite buggy.

I was developing a database as a personal project that involves mmapping a file and the Docker shared filesystem had some peculiar behavior they was different from how normal filesystems work.

That _used_ to be a problem for me, until I found Mutagen. It is _surprisingly_ easy to set up, and my webpack builds in Docker are only about 5-10% slower than running natively.

https://nodewood.com/blog/how-to-speed-up-docker-on-macos/

Sounds like you like Linux but run windows ;) yeah outlook word

Apple (as you may know) did in fact have a spell as open-systemsy Unix-loving guys, starting from the NeXT acquisition but ending no later than the release of the iPhone. The usual pattern, basically: https://news.ycombinator.com/item?id=7525256 .

Ah, yes, the Unixness of the Lisa, the LCII, the Quadra, the System 7, Mac OS 8.1, Mac OS 9.2...

So much Unixness on the Mac OS boot ROM, the Pascal API, the lack of memory protection, and so on...

Please reread my comment.