https://github.com/containers/buildah/blob/main/examples/lig...
Since Dockerfile is a rather limited and (IMHO) poorly executed re-implementation of a shell script, why not use shell directly? Not even bash with coreutils is necessary: even posix sh with busybox can do much more than Dockerfile, and you can use something else (like Python) and take it very far indeed.
Things I've run into:
* Cannot compose together. Suppose I have three packages, A/B/C. I would like to build each package in an image, and also build an image with all three packages installed. I cannot extract functionality into a subroutine. Instead, I need to make a separate build script, add it to the image, and run it in the build.
* Easy to have unintentional image bloat. The obvious way to install a package in a debian-based container is with `RUN apt-get update` followed by `RUN apt-get install FOO`. However, this causes the `/var/lib/apt/lists` directory to be included in the downloaded images, even if `RUN rm -rf /var/lib/apt/lists/` is included in the Dockerfile. In order to avoid bloating the image, the all three steps of update/install/rm must be in a single RUN command.
Cannot mark commands as order-independent. If I am installing N different packages
* Cannot do a dry run. There is no command that will tell you if an image is up-to-date with the current Dockerfile, and what stages must be rebuilt to bring it up to date.
* Must be sequestered away in a subdirectory. Anything that is in the directory of the dockerfile is treated as part of the build context, and is copied to the docker server. Having a Dockerfile in a top-level source directory will cause all docker commands to grind to a halt. (Gee, if only there were an explicit ADD command indicating which files are actually needed.)
* Must NOT be sequestered away in a subdirectory. The dockerfile may only add files to the image if they are contained in the dockerfile's directory.
* No support for symlinks. Symlinks are the obvious way to avoid the contradiction in the previous two bullet points, but are not allowed. Instead, you must re-structure your entire project based on whether docker requires a file. (The documented reason for this is that the target of a symlink can change. If applied consistently, I might accept this reasoning, but the ADD command can download from a URL. Pretending that symlinks are somehow less consistent than a remote resource is ridiculous.)
* Requires periodic cleanup. A failed build command results in a container left in an exited state. This occurs even if the build occurred in a command that explicitly tries to avoid leaving containers running. (e.g. "docker run --rm", where the image must be built before running.)
In case you were curious/interested, docker has the ability to load context from a tar stream, which I find infinitely helpful for "Dockerfile-only" builds since there's no reason to copy the current directory when there is no ADD or COPY it is going to use. Or, if it's a simple file it can still be faster
tar -cf - Dockerfile requirements.txt | docker build -t mything -
# or, to address your other point
tar -cf - docker/Dockerfile go.mod go.sum | docker build -t mything -f docker/Dockerfile -Though, either approach does have the tremendous downside of needing to maintain two entirely separate lists of the same set of files. It needs to be maintained both in the Dockerfile, and in the arguments provided to tar.
The actual reason is that build contexts are implemented as tarballing the current directory, and tarballs don't support symlinks.
Err, don't they? If I make a tarball of a directory that contains a symlink, then the tarball can be unpacked to reproduce the same symlink. If I want the archive to contain the pointed-to file, I can use the -h (or --dereference) flag.
There are valid arguments that symlinks allow recursive structures, or that symlinks may point to a different location when resolved by the docker server, and that would make it difficult to reproduce the build context after transferring it. I tend to see that as a sign that the docker client really, really should be parsing the dockerfile in order to only provide the files that are actually used. (Or better yet, shouldn't be tarballing up the build context just to send it to a server process on the same machine.)
In other words you can use this code to catch this error:
ARG DEBVER="10"
ARG CAPVER="7.8"
FROM debian:${DEBVER}
RUN <<EOF
set -eu
printf "DEB=${DEBVER}\nCAP=${CAPVER}\n"
EOF
SHELL ["/bin/sh", "-o", "nounset", "-c"]
RUN printf "DEB=${DEBVER}\nCAP=${CAPVER}\n"In general, I think of multi-stage builds as if multiple docker files were concatenated; if ARG or SHELL doesn't exist in the theoretical split Dockerfile, then it wouldn't exist in the multistage build.
Docker(file) is a thing which brought in some quite neat new ideas, and a lot of these "footguns". I did read the docs, and I am comfortable now building even complicated "Dockerfile-architectures" with proper linting, watching out for good caching, etc. I wish there were better alternatives; I saw images being built with Bazel, and thanks, but that is even more convoluted, with presumably a different set of footguns.
I am not saying that Dockerfiles are good. But I also think that knowing the tool you use differentiates you from being a rookie to an experienced professional.
I agree in general, though I'd also add that a tool requiring somebody to know and avoid a large number of footguns is what differentiates a tool from being a prototype and being production-ready. While Docker has a large number of features, the number of gotchas makes it feel like something that should only be used in hobby projects, never in production.
In the end, I ended up making several scripts to automate the generation and execution of dockerfiles for my common use cases. I trust myself to proceed cautiously around the beartraps once. I don't trust myself to sprint through a field of beartraps if there's a short deadline.
https://www.docker.com/blog/introduction-to-heredocs-in-dock...
It would still be possible to accidentally reference uninitialized environment variables in contexts other than RUN though. Having those be treated as errors would be useful.
Better defaults and/or an easier way to change the defaults would be good!
The previous commenter is right -- it's all very well to say "just read all the docs" or "it's consistent with the way other tools work", but sometimes both the docs and the tools can be made less error-prone for users.
Environment variables can be useful. The first version of some code can be oblivious to environment variables and the second version can use them for feature flags. Because it is entirely optional to provide these, the second version of the code can be deployed to the first version’s infrastructure, and the first version will happily ignore environment variables set in the second version’s infrastructure.
As a tool for incrementally moving heterogeneous parts of a system forward without needing atomic leaps, the use of environment variables as optional flags is very useful.
I just wouldn’t even start from there. Instead I configure everything in code, in one repo, with all new deployments appearing as new clusters where the only thing they have in common is that they take production traffic. This kind of production infra is very useful but there are downsides for sure. It makes database schema changes, for example, something between very hard and anathema to execute.
Perl has “use warnings ‘uninitialized’” for the same reason.
It sounds like this feature would be worth considering for the Dockerfile spec.