There were two major obstacles; the file system driver and the Docker daemon. I ended up settling on 1.11.{i forget}, because virtually every other version was unusable. As sad as it may sounds, I was tracking two metrics; probability the container will start and time until the docker daemon reaches a state of irrecoverable dead-lock.

Personally I found containers (on a highly contentious system) launched at ~98% success rate with a time-to-dead-lock somewhere around 6500 container start/stop (it was a pretty fat tailed distribution). For a busy system... those numbers equate to an administrative headache.

And on Ubuntu/Debian the defaults (the thing the majority of non-specialists are probably using) were far worse. The only filesystem driver which seemed to work at all was devicemapper direct-lvm.

-

If you have developers on your team who can spend their time figuring out the one magic incantation that makes Docker work most of the time, you're fine. Everybody else should follow his advice for using service providers.

Admittedly, the latter strategy is a bit frustrating and counterintuitive on 16.04. When you install Docker it will start automatically and hang because it can't find the AUFS driver and Device Mapper isn't set up. The solution is to either 1. modify policy-rc.d to prevent services from automatically starting or 2. set up Device Mapper (install dmsetup and run `dmsetup mknodes`) before installing Docker and changing the storage driver. Unfortunately, these workarounds are not particularly well-documented.

I would be interested to see how Docker 1.12 or 1.13 with a modern kernel and OverlayFS would be able to handle your described workload.

FYI: It's because of this sort of bullshit that there are articles called "Docker in Production: An history of failure".

And there goes my hope of ever running Docker on Ubuntu with any success.

I no longer work with that company and so I can't tell how those improvements would change the reliability. Presently I am using docker with btrfs on a low-throughput workload and it seems to work just fine.

Tangentially related - I have been using OverlayFS without Docker for about a year now. It's pretty great. My read-layers can be on an NFS drive and the penalty for writes is relatively small. It basically gives me 90% of what I wanted from Docker in the first place.

The guy works in finance, not an area where you can typically say "oh, no worries, we'll just push out this distro with a non-standard kernel." Enterprise IT, and specifically finance (and healthcare) have different requirements. Currently working with a client where not only the full technology stack has to be certified by accountable vendors, but all the internal documentation and customer facing content has to pass strict legal reviews.

"Here is Debian with a custom kernel" won't fly. Nether will "Here is Debian" - Ubuntu LTS will barely pass the line, and that will be after a whole lot of timewasting and ass-covering from many people involved.

Absolutely right.

> Docker Issue: Breaking changes and regressions

Yes, it's new, and it changes regularly. This is why you do what you do with any other piece of software in an enterprise environment: Pick a stable release (e.g. 1.7), deploy it, then upgrade your development environment to the next stable release (e.g. 1.11) and work through the breaking changes. Regressions are frustrating, I'll agree, but it's free software: report the regressions and help the community fix them.

> Docker Issue: Can’t clean old images

A built-in feature to do so was added in 1.13 (a few months after this article was published).

> The only way to clean space is to run this hack, preferably in cron every day: docker images -q -a | xargs --no-run-if-empty docker rmi

That's not a hack, that's how you do things in Unix.

> As a long-standing goal, the AUFS filesystem was finally dropped in kernel version 4.

> There is no unofficial patch to support it, there is no optional module, there is no backport whatsoever, nothing. AUFS is entirely gone.

While the first point is technically true (actually, based on some light googling, I'm not sure it was ever merged in the first place), many distributions provide it as an optional kernel module. For example, Ubuntu provides it in linux-image-extra. Yes, Virginia, you can build kernel modules from source.

> How does docker work without AUFS then? Well, it doesn’t.

It does: Btrfs, Device Mapper, ZFS...

> So, the docker guys wrote a new filesystem, called overlay. [..] Note that it’s not backported to existing distributions. Docker never cared about [backward] compatibility.

Docker supports multiple storage drivers, one benefit of which is to be able to support older systems: AUFS on older distributions, OverlayFS on newer. The container abstraction allows you to not care about the underlying storage subsystem.

> Right now. We don’t know of ANY combination that is stable

In November 2016? Are you kidding?

0. https://thehftguy.com/2016/11/01/docker-in-production-an-his...

wget -q https://raw.githubusercontent.com/docker/docker/master/contr... -O - | bash