- it's usually headless
- it comes with a layer that mounts the host file system, instead of installing extensions
- better testing of that layer on all platforms, especially the ones that need to add a Linux kernel? (Windows and Mac)
- it's harder to ssh into a container, manually fix things and persist the changes without updating the Dockerfile. We can do that with a Vagrant machine.
Anything else?
Perhaps the difference is that storing your changes is an explicit operation that needs to happen consciously, and that data is usually persisted outside of the container.
That's more difficult to happen with a container and its image.
On the other hand, to have actual reproducibility you need to self build and self host everything down from base userland. However, once you achieve that, reproducible machines are one `apt install devenv` away.
What docker/containers do and excel at, compared to traditional workstation provisioning, is reduction of dependency trees via isolation. With one single user process running dependency trees are shaken and chances of accidental dependency collision drop. Does this count as solving dependency problem? Personally, I say no, but understand the other side of the argument.
And being able to make hermetic environments for integration tests used to be almost impossible, and today, depending on your stack, it is trivial with libraries like testcontainers.
It really does come down to the concepts that underlie OCI images and Dockerfiles. Why?
- Immutable root design promotes hygienic containers that keep state only in mounts. This makes upgrading, backup, etc. of containers very easy. Containerized software can be written around the assumption that this is how things work, by having a way to e.g. run database migrations at startup.
- Composable images: Rather than having every environment ever made try to perfect making a reproducible Go development setup, you can just use FROM statements to use other images. With multi-stage builds you can even compose multiple images together to an extent.
- Single-process design: Prior to containers, most VMs operated like full machines that could standalone, with just emulated or paravirtual hardware. Containers are designed around running a single process. In practice, this dramatically lowers the complexity of containers and their lifecycle. Of course, you Can and sometimes even will need to have multiple processes in containers sometimes, but the general mantra holds well.
- Composable containers: Because of the previous point, it's natural to compose containers too, using features like Podman or Kubernetes pods, or Docker Compose. Containers in the same pod can share resources that would otherwise be isolated across typical containers, such as a network namespace. This makes for concepts like sidecars that allow composable functionality, and the ability to e.g. re-use PostgreSQL and Redis containers.
- As a bonus, these approaches avoid plenty of typical pitfalls of configuration management systems often used with e.g. Vagrant such as Ansible. Because the environment is built up from scratch to your specifications, rather than working backwards from a mutable Linux system and trying to bring it to match a certain specification. Some of the problems that can lead to configuration drift still exist, but they are somewhat contained, especially if you use tags and/or image digests to pin your images.
Virtual machines are technically fine, but containers elegantly offer solutions for both development and production deployment issues that VM setups never really did prior.
Indeed, you could make something like testcontainers but for VMs... but then what do you use in place of OCI images and Dockerfiles for the base images? And you need to handle networking between VMs and the host, probably across multiple different VM runtimes. Eventually you realize you're just making worse containers!
Much of what you talk about is tooling and patterns around containers, not containers themselves, such as networking and volume mounts. I would also point out that there are better options to Dockerfiles for containers these days as well, that unlock the real potential of buildkit.
We moved from "here is an (incomplete) description of what I did", which rounds to "please guess all the previous stuff that I did to my environment, before even starting to work on getting my instructions to work" to "download this binary and execute these 5 lines". The incompleteness problem of the description of _the actual thing you want to do_ hasn't gone away, but the largely not even mentioned description of where you have to do the thing... that has been replaced by Dockerfiles
If you think Dockerfiles are great, then you haven't hit the day 2 problems yet or explored the better alternatives for defining containers.
Also, the whole point of the original post is to point out that what you are saying is not magically true for containers
The reason why containers are good is entirely about the concepts, NOT the actual engine or technologies. Again, you can literally run OCI images in a virtual machine; Ignite will run them in a KVM. That's as literal as you can get.
Docker existed and did a lot of the same things it does today in 2013.
The ONLY thing docker did was make a small amount of questionable UX around the low-level container primitives. They did not invent anything, definitely not containers. I could share VM images before Docker, nothing new there... the concepts all existed before containers.
You can literally run... anything... on a VM, why make a point about being able to run OCI images? This is like a no-op.
I feel like we took Ansible roles (for example, it could be Puppet, CFEngine, whatever) and spread them in uncountable and not reusable Dockefiles and bash scripts (oh the irony). But we still have the config managers too, because who could have imagined, you still have to have bare metal underneath.
Docker (like every other tool before it) started nice, clean and simple, because it didn't cover all the real needs. As those where added on we ended up with tens of tools on top of k8s and now here we are in yaml hell with version compatibility still not solved. And a new generation will come along and repeat it with another set of tools becase "this time it will be different". You can not get rid of complexity, you can only move it from here to there. And if there == YAML then may God have mercy on you.
They both achieve the same aims - reproducible setup - but (on Linux) a container is uses less resources than a VM, so it's lighter to run them both side by side. This matters for developer machine setup.
When it comes to production deployments, I haven't even started looking at k8s because I don't need that level of complexity.
No, you cannot remove it, but you can definitely add more. Every time I have to work with the modern Javascript tooling upon tooling upon tooling, I want to punch a baby rabbit.
If it worked, it worked, though. I'm not arguing with results.
What we actually did, back in the day, was host our own yum repo and build RPM packages. For the most part, the only differences between the nodes were the installed packages. But having to define 100s of node types in puppet and force every developer to know at least some puppet, and troubleshoot failed runs, and understand RPM packaging foot-guns (like, if you screw up your package uninstall scripts or file lists you can make it impossible to remove a package, or leave unwanted files behind, meaning that somebody has to go and fix all the "pet servers" or rebuild them).
Fundamentally, this also leans towards long-lived servers that you tend to and care for. We had teams with long-lived servers that deployed incremental app updates every few days. You know how many times we tried to build a node from scratch, only to find out that it failed to build our didn't work after being built? Eg, some historical package gets removed, leaves files behind, so everything still works. But on a clean install it doesn't, so you end up on a sev1 incident using strace, ldd, hunting in logs etc etc.
You also wouldn't believe the number of incredibly stupid screwups that broke dozens of servers at a time when developers were building their own RPM packages and using Puppet to deploy them. What made docker so good was that developers weren't given the ability to package up something into an RPM that gets installed by root, potentially fucking up the host. And building a container either works or doesn't. Or forced our developers, for the first time ever, to know 100% of the steps required to get the software to work in isolation. And instant rollback meant running the previous container tag. Honestly, the puppet > docker change was the biggest leap in reliability and simplification that I ever experienced.
Building an image from the ground up is the other direction. The difference is subtle, but if you have an exact known starting point, then declarative configuration management is not only unnecessary, it's downright worse than just running a few commands to get to the desired state.
The proof for me is in the pudding. Container building is a superset of configuration management. You can definitely use Ansible or what have you to build a container root image, but it would be long, overcomplicated, and more fragile than just starting from a known state and mutating from there.
I recently contributed to the Linux kernel and they often get irate over the contributor not following manual steps. They could automate a lot of it with basic features like CI, and your suggestion that it is easy to make things reproducible if you just follow a list of instructions is part of this problem. "Just follow a list of instructions" will never be a solution to a bad workflow, and it is no replacement for a good methodology.
If you do not force best practices in the tool, it permits mistakes. Something you probably don't want to allow in a tool that builds software and manages the software supply chain. Docker doesn't provide a language for describing things correctly, you can make all the mistakes you want to make. Nix, for example, is a domain specific language which won't permit you to commit crimes like not pinning your dependencies, at least in 2023 with pure evaluation mode (on by default in flakes).
> They list the steps that you need to build a working image.
No they don't. They typically list the steps that you need to build a working image yesterday, not today, or in 5 years, which is a very important thing to be aware of, otherwise you might assume the instructions in the Dockerfile were crafted with any intention of working tomorrow. There's no reason to believe this is true, unless you know the author really did follow your list of suggestions.
Nix crashes when you make mistakes in your .nix expression. `docker build` won't crash when you make mistakes in your build, it is unaware and doesn't enforce a reproducibility methodology like Nix outlines in the thesis, an obvious example being the unconditional internet access given by the Docker "sandbox".
Docker does not make distinctions between fetching source code and operating/building that source code. In Nix these happen in two separate steps, and you can't accidentally implement your build instructions in the same step as fetching your program, which would otherwise lead you to execute the build on untrusted/unexpected input. This is just one part of the methodology outlined in the Nix thesis.
TL;DR Nix doesn't suggest you follow a list of instructions to make things reproducible, it just doesn't permit a lot of common mistakes that lead to unreproducibility such as not pinning or hashing inputs.
I'll go as far as to say that if you want reproducible images that don't unexpectedly break without some kind of ability to trace it back to a change, always use sha256 digests in your `FROM` clauses, never fetch files directly from URLs (and/or check sha256 digests of things that you do fetch,) be thoughtful about the way your container is designed, and favor multi-stage builds and other OCI image builders to writing silly shell scripts that wrangle the build command in unusual ways.
But generally? It's still a massive improvement just having the steps written out somewhere. When I first started, it seemed the best you could get is a Vagrantfile that basically worked after you stepped on a bunch of rakes to figure out exactly what quirks you had to work around. With containers, things break a lot more predictably in my experience.