Or maybe we can realize that npm and webpack are both trash. And we should stop embracing them and move web dev away from this mess.

Don't get me wrong. They were good workaround years ago. But I hope we can do better.

Creating and mounting a loop-volume within the HGFS and then running NPM in there, would probably also be helpful. It certainly is when wrangling a directory with lots of tiny inodes over SMB. (It’s also what Apple themselves do for running Time Machine over SMB: they create a sparsebundle image on the remote and mount it locally, and write to that.)

(The efficient thing about this class of solutions, if you’re wondering, is that the client mounting the loop-image ends up owning + managing the loop-image’s internal filesystem’s metadata within its own local disk cache, such that it can coalesce filesystem metadata writes and only push a new copy of the disk-image blocks backing the filesystem metadata after a potentially-huge number of changes. With a network/host-guest filesystem, meanwhile, every filesystem metadata change must become its own synchronous message to the host, to be pushed to the host’s filesystem driver for linearization, so it can succeed or fail relative to other things going on within the host.)

I used to run my own Xhyve based VM just to avoid the brutal VirtualBox Docker system on macOS. Man that’s some bad memories.

The newer Docker for Mac is better, but file system perf still could be massively improved. Big node_module directories can still cause pain even today

> Big node_module directories can still cause pain

is there any other kind?

Bigger node_modules directories

Yep, using Webpack in Docker on a Mac is essentially unusable. I see around a 5-10x slowdown in Webpack builds. Using docker-sync seems to help the CPU out a bit, so my MacBook Pro doesn't do the jet engine space heater routine quite as much, but the performance is still nearly unusable.

https://docker-sync.readthedocs.io/en/latest/

webpack can be made use-able relatively easily if you do not mount node modules over the shared file system. I've been doing this for quite awhile with a volumes declaration in docker-compose that looks like this (running nextjs, assuming /usr/src/app is where your dockerfile has your node stuff):

(on service definition): volumes: - .:/usr/src/app:cached - node_modules:/usr/src/app/node_modules/ - next_artifacts:/usr/src/app/.next/

and then in the top level volumes key defining node_modules and next_artifacts as blank/default.

That means I mount everything except node modules and the build artifacts so the shared filesystem does a LOT less work trying to sync stuff. The downside, of course, is that I need to run npm commands both inside the container and outside if i want them in my IDE. A fair trade for decent performance. That setup is still not as fast as native but definitely usable and does not send my machine into space header mode much more than normal usage.

Interesting. I have always had the same docker-compose setup with a separate node_modules volume inside Docker, but I've always still had the jet engine space heater issue.

I've been able to improve performance in docker-sync by adding high-churn folders that I don't care about to sync_excludes

Virtualbox-backed Docker on Windows was my first introduction to Docker - it literally put me off touching Docker for 1-2 years!

After eventually falling in love with it on Windows and Linux, I later tried it on MacOS... oh my. Not fun!