shykes
Red Hat was frustrated with us because they were used to calling the shots. They couldn't fathom that we would refuse a pull request from them. Well guess what, every project is entitled to saying no. And Red Hat's contributions were just overall not good, or they didn't bother to consult with the maintainers on our design principles - like independence from the underlying OS distro.
You're reminding me why working with that company was such a pain. Always with the fucking superiority complex and passive aggressive drama.
As for a "security disaster waiting to happen". My view was, and remains, that the only isolation you can trust is the hypervisor. Kernel level isolation is not strong enough, and unprivileged user namespaces never actually mattered in cloud architectures. Your work on that feature was just security theater and ultimately useless, so spare me the lecture.
I do think Docker embraced user namespaces after my time but I couldn't care less.
And yes the community has always been the best part of Docker...
What are you up to these days?
I did speak to Greg several times.
I think over the years there was natural overlap, people who contributed to Docker would also be kernel contributors. Especially when we spun out runc, containerd, which created a more vendor-neutral space for maintainers to focus on the bottom of the userland stack with minimum drama.
To the extent that Red Hat employees overlap with kernel folks, that made it more difficult to collaborate since Red Hat was overtly aggressive towards Docker, and generally made our engineers feel unwelcome on "Red Hat turf". I discovered that they had devolved into a toxic company with a facade of benevolence. This made me sad because Red Hat was an inspiration to me. One of those "don't meet your heroes" situations.
Not sure what you meant by the "ignoring" part.
I hope the plug is appropriate in this setting - apologies if not.
> One use case I've found for it besides CI was building an eval sandbox for testing different coding agents' performance using CLIs, APIs, and coding tasks.
Very nice! Would you be willing to do a video interview and show a demo for the community?
In my opinion evals and CI are converging into a common execution and data model for all pipelines involved in shipping your product. It's Continuous Engineering :)
> while the API and CLI are beautifully made, the documentation and the webapp scare the normies and could be improved.
Thanks for the feedback! Yes, we are working on the docs, it's not perfect yet but trending in the right direction I think. We are also betting that everyone will soon be asking their agent to setup Dagger, so we are investing heavily in making sure agents understand and appreciate Dagger, so they can set it up properly and explain it to their drivers. What a crazy world we live in!
Podman has never been commercialized.
Docker's difficulties in commercializing were self-inflicted. Enterprise and open source teams were culturally siloed and as a result the enterprise products were not "native". There was a lack of focus and a "checkbox" approach to building product. There was no singular "dictator" who could enforce a coherent product design and vision, and it showed. etc.
There's a fun little easter egg: the combination of "Steve Wozniak" and "Boring" is explicitly forbidden, because Steve Wozniak is not boring :) https://github.com/moby/moby/pull/4902
How did I not know about this video until today!
Hope you're doing well.
When self-hosting Dagger, you can run it on Kubernetes. Beyond that, we don't use Kubernetes at all ourselves.
The most successful project that we worked on together was "project Piñata" which later became Docker for Mac. It turns out bundling a Linux VM in a native Mac app, and integrating it into the host filesystem, networking stack etc requires lots of magic..
Now, most of the Unikernel team were very active OCaml developers, and created MirageOS among other ocaml projects. While working on Piñata, they simply reached for the tools they were most familiar with.
I also feel lucky that people are still using my work, that it endures without me, and that it generates (mostly) positive reactions. There was a time where Docker had a lot of haters, which was hard on me. But with time I've noticed that the negativity gets mostly washed away, and what remains is positive interactions like the ones in this whole thread. So now Docker is this cool experience I share with lots of people around the world, and when we meet we get to talk about this shared interest. I love it.
However Moby was a poor choice for a skunkworks project, it involved mostly communications and project management, which requires consistency and scale. Overall it was poorly executed because I failed to build enough support internally ("Solomon's at it again, let him do his thing"). Here I am trying to explain it on Twitter after the keynote, using a picture of a hand-drawn diagram, because nobody in marketing or product was assigned to help. https://x.com/solomonstre/status/855918630915133440
I think the goal of Moby was fundamentally correct: we were trying to replicate the Red Hat model, with well-scoped upstream components (linux -> runc, containerd..) integrated into a commercial product (Docker -> RHEL), with a faster-moving community project in between (Fedora -> Moby).
This was a way for us to clarify that Docker was a product, with a community of users, and Moby was an open source project, with a community of contributors. This clarity was very much needed, because different communities were projecting very different expectations on the brand "Docker", and these expectations conflicted with each other.
In a way, Moby aspired to be what CNCF ended up becoming - the "big tent" where the broader community meets to hammer out the general shape of the container stack. It seems obvious now that a single company can't be the sole authority over such a vast scope. But in the first years of Docker, we really did have that level of influence, because we literally invented the market, and brought the community together in meetups and an ever-growing Dockercon. It's only when Google and Red Hat pushed Kubernetes and CNCF, that our influence over the stack started to wane. Maybe it was for the better, I don't know if we would have been up to the task as designers and engineers. But at least there would have been a consistent design, with a Linus Torvalds figure willing and able to say "no" to stupid and redundant features. For a while I played that role - but then I was replaced by SIGs and committees.
Another explicit goal was to facilitate contribution. We thought Go syntax was "vanilla" enough to attract a broader set of contributors than a more niche, opinionated language like Erlang, Haskell, Clojure etc. I think that also worked out well - Docker was known for attracting many contributors, many of them making their very first open source contributions. We even ran special meetups to help people become contributors, that was fun.
> the previous generation of the technology used by dotCloud [...], which was using LXC with AUFS
One small note, the earliest versions (2007-2011) were based on vserver then openvz, which predated lxc. Back then you had to patch the kernel to get container primitives. And since virtualization also required patches (eg. Xen), we had to carefully combine both sets of patches, which was poorly supported because nobody else was crazy enough to stack 2 kinds of virtualization!
That day is forever engraved in my memory... Off the top of my head:
I signed up for a lightning talk expecting an audience of 10-20 people, for a confidential demo of our unfinished project that we had so far only showed in 1-1 demos. Imagine my surprise when I learned Pycon does lightning talks on the main stage!
The previous speaker didn't show, so I started early. In the recording you can hear me joke that perhaps I should get his time allocation too - the MC was not amused.
I went over time and had to speedtalk through the end.. The MC once again was not amused and practically shoved me off stage :-D
We had a small sponsor booth at the even, where our small marketing team was prepared to pitch our product (DotCloud) but was thoroughly confused when a horde of people showed up after the talk, to ask about "Docker" - which as far as they knew was the founder's side project.
I drove down to Santa Clara on my newly acquired motorcycle - my very first trip on the freeway. I remember telling myself, "today is the day to take all the risks!"
One design principle that was very important to me, was portability and avoiding fragmentation. I wanted every image name to have the same meaning in every engine, so that when writing a Dockerfile, you could have confidence that it would actually behave the same on every docker installation. In this we disagreed with other platforms (like Red Hat) who sought to "embrace and extend" Docker by changing the meaning of short names. We always fought this to avoid fragmentation of the platform. This did not make sense from Red Hat's perspective, since they were the platform - who cared about fragmenting Docker? That led to some negativity which is probably what you're remembering.
At the same time early Docker users wanted the ability to host their own images without being forced to use our central registry. Using Go's system solved both problems at once.
We actually use the same Discord server for community and private team communications. So we are literally there all the time.
Yes, I still use Docker. Although my way of using it has changed over time, like many others I assume.
My new project, Dagger, although it doesn't directly compete with Docker, does implement its own container runtime and orchestrator (for CI pipelines), and we recently added compatibility with Dockerfiles and Docker Compose files... So increasingly I find myself using Dagger instead of Docker, in the context of my CI pipelines.
On the other hand, I use docker compose on almost every personal project - like a minecraft server for my kids, home automation, AI hacks, etc. These days I just let my AI agent set it up for me, though :)
I did not start a band, but I did build a nice house, help my family etc.
I still program everyday, working on my second startup, https://dagger.io (please forgive the plug). We have a very active discord if you want to come say hi.
Docker in its earlier incarnations was based on openvz, which predates lxc. The lxc project was initially a kernel development project: a collection of kernel patches to add container support. The userland tool was just a companion. When it became apparent that these patches would "win" the race to be upstreamed even though openvz was much more mature, we switched our tooling to lxc. The patches matured over time, but the userland tool (what we now know as "lxc") was finicky.
The original open-source release of Docker wrapped lxc for low-level container operations. But it was just too unreliable and we swapped it out for an in-house Go implementation called "libcontainer".
As for "commercializing": Docker was famously bad at commercializing anything, we open sourced so much (maybe too much) and still people complained that we were not open enough.