While that's true for processes running in containers, building containers is generally noticeably slower than Makefiles as it involves several extra steps besides executing instructions (e.g. loading build context, gziping files, calculating hashes etc...)

BuildKit(Docker build engine) maintainer here. It’s definitely our goal to limit the friction, so you get the benefits of containers but with similar performance as running things on the host. For the points you listed: BuildKit sends context incrementally between builds that transfers only deltas and not the whole context. Gzip is only needed when you want to push the image to a registry that doesn’t happen as often as your dev builds. For the hashes, we do need to calculate some for build cache (that of course makes your builds faster than they would be without it). For some bigger objects like layers we can defer checksum computation to push phase similarly to compression.

Another thing we have done is added cache mounts that let you keep application-specific cache between builds. This is usually, where the speed benefit from running things on the host used to come if your tools wrote some cache under $HOME etc.

There is still a constant container sandbox initialization time ~100ms, with some room to optimize that as well. If you have good examples of builds that can’t be easily optimized in containers would love to get your feedback/examples on the issue tracker.

You'd typically not rebuild CI images that often.

If you're following the article's suggestion of using docker to replace makefiles and/or run build processes in docker and just spit out the final product using --output, the you will build the image on every single compile. And I'd try to give docker credit for caching all the layers it can, but make caches results too so it's not necessarily even winning there.

You’re building Continuous Integration images all the time. The more often the better. Hence the aspirational “continuous” in the name.

> building containers is generally noticeably slower than Makefiles

I totally agree, if the year was 1992.

Compute is cheap. Storage is cheap. Let's not try and keep living in the past.

No, it’s noticeably slower than Makefiles on present-year hardware. This degrades the developer experience regardless of how cheap compute and storage are. (This is mostly Docker’s fault and not inherent to containers, though.)

Part of CI is fast feedback. Slow builds started soon still suffer from not providing the full experience.

It’s not a ratio that’s a problem, although that doesn’t help. It’s that there are psychological cutoffs where adding a minute to a build drastically changes how people interact with it. 1 vs 3 vs 7 vs >7 minutes make substantial differences.

So taking a build from 8 to 6 minutes is valuable out of all proportion to the actual speed up, and slowing from 6 to 8 is a substantial loss.

> This degrades the developer experience regardless of how cheap compute and storage are.

I would argue that if you only have one good tool to do something, you have an opportunity to make much better tools for that something.

It's not docker's job to be better than Makefile with every use-case. You do realize what you're asking for here right? It's impossible. No tool is perfect for every use-case.

> It's not docker's job to be better than Makefile with every use-case. You do realize what you're asking for here right? It's impossible. No tool is perfect for every use-case.

The root of this thread is

> We actually use docker multistage build as mentioned here as a replacement for make file.

So yes, it absolutely is reasonable to ask that docker, when used in the exact same way as make, should not give us a noticeable performance regression compared to running make.

Did you even read the article?

345 words is barely an article. What about my comments are off topic exactly...?

> You don't want your developers developing on a different OS than your CI & deployment environment anyway

Eh, this kinda depends on what tech you're using. We use node.js at work which has very good cross-platform support. And we've had literally zero issues developing on mac and deploying to Linux in the 3 years I've been here.

MacOS has a case insensitive filesystem (default). If you mix up the case with your files/folders and imports then everything will work just fine on your machine. We had a coworker that did this once. Stupidly, it even passed staging and somehow was merged upstream.

Indeed. This is the one incompatibility I have seen in the wild. On my previous machine I actually created a secondary case-sensitive partition to store code on. But unfortunately our iOS app doesn’t work on a case-sensitive file system so this meant having two different project folders. On my new machine I’ve not bothered and I’m just careful about casing.

Your (potential) problems will come at the installing node/packages (that might use C extensions or whatever) level, not developing on top of it.

Or just tooling issues like BSD vs. GNU versions, Homebrew installing things differently (e.g. using the more logical 'openapi-generator' vs. 'openapi-generator-cli' used by upstream and respected by at least apt, apk, and pacman).

Do you use Python by any chance? Node.js native modules typically bundle the native code as part of the module, and don’t depend on externally installed tools. I’ve never used OpenAPI generator, but that looks true of this npm package too https://www.npmjs.com/package/@openapitools/openapi-generato... (read me mentions the tool is downloaded and installed on first run).

I’ve had issues where modules didn’t run when upgrading OS versions (either macOS or Linux - but usually macOS), but we run prod on LTS versions of Ubuntu so that doesn’t happen very frequently, gets caught by our staging server, and is usually a case of simply upgrading libraries to newer versions.

I use both, and have had such issues with both.

I suppose using busybox as your base image will most readily highlight any missing library issues if you fancy it.

(If I remember in the morning I think I can quite quickly find whatever it was that was last a problem - it was definitely an npm not a pip install when last I encountered it.)

Anecdotally as well, we've had two that I can think of in the last two years (both were specific to Windows).

You can argue or disagree all day long about what OS is best for development, but it's not reasonable to think that developers on MacOS would only write code for MacOS servers.

You can very successfully write code that runs on Linux on either MacOS or Windows, depending on what you're developing.

Probably not a big deal, but just seems pointlessly narrow to think all developers would or should prefer actually developing on Linux exclusively.

I got frustrated enough with Docker on macOS that I now have an Intel NUC to use as my "local development". I still use my Mac for development (and all the tools I'm familiar with and do not want to learn replacements for) while also having "fast" Docker on my desk (and even when I'm not at my desk, Docker-on-Linux over VPN annoys me less that Docker on macOS).

Feels kinda wasteful sometimes to have all this compute horsepower in my MacBook, and then offload a bunch of the workload to a few hundred bucks worth of external hardware, but I reckon it'd saved my boss the entire cost of the NUC in the first week or two.

Not the right answer for everyone, but it works for me.

It's really not that bad of an idea for anyone, I can only speak for VSCode but what you're talking about isn't a huge gap from WSL2.

Except rather than using WSL2 for your local remote (confused yet? I am) sessions.

Hardware _is_ cheap, if MacOS can't support a decent enough environment for Docker, provide your Engineers with some cheap hardware like a NUC or even a remote AWS Linux EC2 instance or similar and get a native *nix environment that way.

You can but it’s easy to get your environment messed up with python.

Personally I’m a fan of buying a fancier Mac and running a VM that resembles production somewhat. I was using VMWare Fusion and now UTM. Works great for me by minimizing Homebrew drama.

> Docker is just as fast as running native on Linux

Once the docker image is pulled into cache, a runc process is started, that process is placed into a specific cgroup and given various isolated namespaces, any volumes are mounted into the file system namespace, any special network mappings are created, and otherwise, yes running within Docker is as fast as running natively.

> You don't want your developers developing on a different OS than your CI & deployment environment anyway.

Who's running OS X servers in production?

What are the devs running OS X and IntelliJ tools deploying on?

We have a pair of macOS servers in prod. But it's a weird edge case (we need to build iOS apps as part of the work queue, so there's no choice).

Mostly our devs run whichever of macOS or Windows is their personal preference (and in a few cases they choose Linux), and deploy to containers (or occasionally bare) Ubuntu or AWS Linux.

We occasionally get new devs bumping into the well known problems (case sensitivity, file path differences, some weird depenacncy that ends up platform-specific), but that's the kinda thing that is very easily debugged by someone who's made those mistakes before and even the greenest of new devs rarely make that same mistake twice (or at least they debug it themselves and don't admit to it).

WSL2 solved docker for us on windows (except broken opencl), so afaict just OS X now unless you go into major contortions

We do both containerized and native for developing one service, but generally docker compose for the composition (vs say k8s or all native). And important that CI is in docker: that means we can do accurate local docker testing of same env in case of CI fails.

> You don't want your developers developing on a different OS than your CI & deployment environment anyway.

It depends what you're working on, but for a classic server backend in a high-level language, I don't find it a problem. It's very rare to hit an OS-specific problem, and it shows up in CI so the dev can correct it before code review.

For me, filesystem I/O always ends up being significantly slower from inside docker containers.

Is there something obvious I may be doing wrong or some resources I should read about this?

It’s been a major issue for me. It seems to happen regardless of whether I’m using Docker Volumes or not.

IO (both network and disk) are afaik taking significantly roundabout way with docker vs bare linux