I wish Dockerfiles allowed you to group commands so that they would end up in the same layer, so you don't have to do something ridiculous like

RUN cmd1 && cmd2 && cmd3 && cmd4

maybe with brackets:

    # 1 layer

    { 

      RUN cmd1

      COPY someshit

      RUN cmd2 -lots -of -long flags

      ...

    }

You might be interested in this: https://github.com/uber/makisu

I've not used it but it's on my list to try out, the explicit caching I think ties in with what you're after:

* Requires no elevated privileges or containerd/Docker daemon, making the build process portable.

* Uses a distributed layer cache to improve performance across a build cluster.

* Provides control over generated layers with a new optional keyword #!COMMIT, reducing the number of layers in images.

* Is Docker compatible. Note, the Dockerfile parser in Makisu is opinionated in some scenarios. More details can be found here.

I can't edit any more, but I'd really be interested in people's experiences with it.

Nowadays docker server, when run with the experimental flag, allows you to build images with the `--squash` option, so your whole image will contain a single layer.

Alternatively you can use buildah [1] to build your images via a script, but you'll lose the cache docker puts in place.

[1] https://buildah.io/

Why do you want to avoid multiple layers. Performance?

Build performance, image size and deployment speed mostly.

Each layer is a write only operation. So if you `RUN some_build.sh` and then `RUN rm junk`, you basically do nothing to reduce the final image size. So you see a lot of like `RUN some_build.sh && rm -rf junk`. There's some overhead with each layer as well.

Size of the resulting container is my understanding.

Hey there, OP here.

Thanks a lot for the feedback. I believe simplicity at this phase of learning is much more valuable than efficiency, that's why I haven't tried to produce the most efficient images as examples. Since your suggestion also simplifies the Dockerfile itself, I have replaced the previous one with your suggestion, thanks!

You do realize that bad habits tend to not die, right? Don't teach bad practices because you're targeting newbies...teach them the best practices from the start.

Agreed, I didn't mean to teach bad practices to anyone, I am just trying to put simplicity in front of efficiency so that people can grasp the technology better and benefit from that.

The article is a “what is docker?” and This is a perfectly fine way to use docker. There’s always room for optimizing layers in the future. Always start with the basics rather than overwhelming people with details.

The point I was making is why the following snippet doesn't make your image smaller _is the basics_:

> RUN echo something > some_file

> RUN rm some_file

Neglecting to even mention it leads to issues down the line because people don't get the fundamentals. There are a lot more confusing things than this when working with Docker.

Small images is one component of optimizing a container. I disagree that a small image is fundamental. In fact I’d say it’s largely a small optimization detail that has very little real world impact.

Really nobody really cares about container sizes except for their own efficiency. We run thousands of containers. Many of them tens of GB. They could all be optimized to a small size but it really doesn’t matter and is not worth the engineering time most of the time.

It's not necessary about the size, and nor is it always a premature/useless optimization. It's about knowing how layers interact which is important. And not hard to teach either.

And yes, if you're adding 5+ GB to each image build through something silly like a 'RUN chown' it's worth fixing. Maybe not complex refactoring, sure, but we are talking about the basics here.

I think their point is that for most use cases, large images don't have a sizeable real world impact. They're more expensive to move around, and you can only fit so many large, distinct layers on a single machine, but a moderately sized organization will barely notice the cost.

They do have a sizeable real world impact. Moderately sized organizations usually build and store the images in a public cloud, which in turn do not provide any straightforward registry cleaning capabilities (at least I didn't find any in AWS or GCP). This leads to hundreds and thousands of gigabytes being wasted and money being flushed down the drain. Not to mention that network traffic in public clouds can quickly rack up the bill as well.

This Docker and Kubernetes hype and the fact that few people understand and care to implement these technologies correctly fills Amazon's and Google's pockets with money that could be spent elsewhere.

> It's not necessary about the size, and nor is it always a premature/useless optimization. It's about knowing how layers interact which is important. And not hard to teach either.

Learning layers is not critical to get up and running with Docker. You can learn it easily afterwards, but on the very first introduction it's just unnecessary cruft that distracts you from your goal.

More importantly, you can actually use Docker in daily practical applications your whole life not caring about layers. Think about it: why should you care about layers if you don't care about build size or build time?

At this stage, caring for layers is just premature optimization.

It’s like having a “What is sorting?” article with bubblesort as the algorithm of focus. It’s easy to explain and answers the question. Yes, time complexity is fundamental to designing algorithms but it’s not something necessary for an introductory article.

That is more the fault of docker than the user. Docker should be smart enough to spot files that exist transiently only for a few layers in a final image, and then maybe decide to never store those intermediate layers (always regenerating them on demand, or storing them only in some less-permanent form)

This article also has an image that shows containers running “on top of” docker; that’s not how any of this works. Docker is just the daemon that sets up the namespaces and runs the containers. It’s not some magical runtime, any more than “iptables” is. It just talks to the kernel.

I think this post is attempting to oversimplify, and is doing a bad job of teaching in the process.

Running Docker containers is definitely on top of the Docker runtime. Maybe not from a PID parent point of view, but absolutely from an abstraction point of view.

Try doing anything hardware-network-topology relevant stuff without `network=host`, like running a ROS cluster with direct ethernet connections between hosts and cameras.

It's definitely "on top" in a way bare-metal isn't. Same way that a vm is "on top of" kvm even though the cpu supports hardware virtualization.

Nvidia-docker is another abstraction. It's a pass-through. It's not the same as bare-metal. Certainly feels the same 98% of the time, but in that 2% it is very much apparent there is a layer in between.