I have always wanted that too.

the way to create small docker image is:

  RUN apt-get install foo && \
    do bar && \
    do more && \
    remove all of the crap you don't want

It should be replaced with actual first-order dockerfile statements

You can use multi-stage builds, but they are sort of hacky and do not match your thinking.

You can also use --squash but it is a hack too and usually is not available where needed because of experimental.

Also, ADD is stupid.

You can add a tar.gz file and it will expand it (but you can't use -k)

You can add the same file using http/https, except it WON'T expand it so back to the RUN dilemma you get:

  RUN apt-get install wget && \
    wget foo.tar.gz && \
    tar xvkf foo.tar.gz -C dir && \
    rm foo.tar.gz

I too have wanted a solution for the massive amount of layers that are generated just by doing normal things. When you reach a point where you're trying to be so clever just to avoid the creation of a useless layer, it's the tooling's fault, not yours.

You might investigate buildah. You can achieve precise control over layers. Also, buildah mount is awesome; direct access to a container file system from the host; COPY/ADD et al. are obviated.

Have your tried multi-stage builds? We use them almost everywhere to slim down images to only have what they need in production.

https://docs.docker.com/develop/develop-images/multistage-bu...

Yeah we asked for transactions and they gave us this... thing.

It’s cute, and useful, but in the end it brings back the primary problem that Docker actually solves: absolutely getting all of the files your code needs to run into the package, and the right version.

Multistage builds require you to airlift files out of one image into another. Accurately. If cherry-picking is the best option on offer by docker, there are plenty of other tools that can do that.

Collapsing layers is about seeing that an image provides a set of services at point A, and another set at point B, and that nothing in between represents an interim state of any substantial value. For most images, this is one or two layers that relate to either the main payload, or one particularly volatile dependency.

The real purpose for multi-stage builds is for compiled languages.

If you have some service written in say C++/Java/C#/Go, then building it in one container (that has the build tools) and copying the build result to another that only has the stuff actually needed at runtime makes good sense. That is much easier and saner than trying to uninstall all the build time components after using them.

It is not like for desktop software we install the build tool, build, and then uninstall build tools. Instead we grab the relevant build outputs and package them into an archive or installer.

But as a solution for clearing out cruft like package manager caches, etc, yeah multi-stage build is not really optimal.

Being able to write the executed steps in a natural way, and then specify at the end to compare everything to come specified previous layer, and create a layer that just has the differences would be really useful.

They could even still cache the intermediate layers as part of the build caching process if they wanted, so long as they are not included in final image as pushed to a repository.

Docker does have a super cut down version of this with its experimental `--squash` option, but that only makes sure there is a single new layer for the output of the whole dockerfile run. You cannot use this if you for some reason want exactly 2 new layers. This limitation is probably acceptable in a lot of cases. However this option is not supported in the buildkit based backend yet.

RUN --mount=source=my_script.sh,target=/tmp/script.sh /tmp/script.sh

This will adds the script without adding a layer for it.

The appropriate solution for this is to put your shell code in a shell script and RUN that.

Then you have to create a layer for sticking the shell script into the container and another to clean up after yourself.

If the Dockerfile is canonical, I should have as much as I can reasonably put into it.

Isn't this the same?

RUN apk update && apk upgrade && apk add foo

In the simple case, yes it is, but it scales poorly; consider ex. https://github.com/docker-library/postgres/blob/b9c080857b88... which is 112 lines run together like that to keep things in a single layer. This is pretty bad for readability. Also, as sibling comment notes, that only works for RUN commands, not other or mixed groups of commands.

It is, but the syntax is uglier. Also imagine that you might want to do more than just RUN in a single layer. Maybe COPY some file into the container and then RUN something that uses it.

Spend some time on docker hub and you'll find thousands of Dockerfiles with RUN <150 lines of && concatentated scripts>. It's dysfunctional.

Good to know, I often wondered why people produced these giant RUN statements instead of making multiple RUN.