Docker is one of many attempts at reactions to just how awful it was to manage things that way. It's not perfect, but the key takeaways is that a lot of people have independently come to the realisations that you wan deployment to as much as possible deploy static copies of already test images rather than run a bunch of configuration steps; we want isolation to various degrees; we want reproducible deployments; we want to be able to easily isolate state and regularly throw away/replace the static elements to prove that the state has been confined to the documented locations and ensure any manual tweaks are removed.

All of this is possible without tools like Docker, but they're a hell of a lot harder without some of the facilities brought by tools like Docker. It doesn't have to be docker, but it does need to be something offering isolation, a mechanism to manage images, and ways of managing data volumes. You can do that in bash if you want to, but the value in these tools is not to have to reinvent it from scratch.

What it would really require is a new application model. With a simple interface with the OS to define all of its needs in term of firewall rules, ports to be opened, access to hardware, etc. And very few interactions beyond that. Storage is cheap, it really doesn't matter to transfer a couple hundred of megabytes of binaries more. You would have a single binary file. And most importantly you would achieve version independence, i.e. the binaries of the app shouldn't even care whether it is running on ubuntu or another distribution (and it could even be cross OS) and mostly what version it is running on.

I can't help thinking that Docker is building a massive legacy application headache further down the line when all these applications will not be developed anymore, the developers will have moved on, the technology stack moved on too to new shiny tools, but the underlying OS needs to be updated and these legacy apps still need to run. The compatible base image will probably not even be available anymore. Some poor guy is going to have a horrible time trying to support that mess.

> With a simple interface with the OS to define all of its needs in term of firewall rules, ports to be opened, access to hardware, etc. And very few interactions beyond that.

That is largely what Docker is providing. It doesn't do it perfectly, but it does do it.

> I am can't help thinking that Docker is building a massive legacy application headache further down the line

That "legacy application headache" existed before Docker too, and it was much worse, because you had to hunt down toolchains and underlying OS images that'd let you get the apps to compile or run, and you could often not rely on the build and installation instructions to be complete and precise enough, because you had no guarantee that nobody would manually interfere with the process. This is in my opinion the biggest thing Docker's focus on automated image builds have given us: if you have a remotely sane process, the Dockerfile serve as evidence that the build steps work unattended (yes, you can do this with CI to verify your build steps too... until they fail in production and someone applies a workaround and don't roll it back into the build scripts; it's possible to add all kinds of steps to avoid it, or you can just build to images - docker or otherwise - and prevent the problem entirely by overwriting all he static data on every deployment)

> The compatible base image will probably not even be available anymore.

If you have a running copy of the image, then Dockers layering mean you have the compatible base image. Won't necessarily help you that much when you want to upgrade to a newer version, but at least it allows you to inspect what the dependencies are.

But all you have done is to move the problem further away from the ops guys. You still need your tool chain and underlying OS images and everything else to rebuild and re-image your guest image. When the dev team will be gone, that's not going to be any easier to maintain. Think about today's legacy software, developed 10 or 15 years ago. Even if you have the source code the tools to compile it probably don't even compile on today's OS versions (if you can even get the right licenses).

A docker with no coupling between the host and guest OS would make me happier. Then you have a binary you can just deploy 10 years after it has been compiled. And then you could extend this model to all kind of apps, including client/UI apps.

You're not going to be able to escape the problem that you need to update the dependencies.

> But all you have done is to move the problem further away from the ops guys.

No, what you've done is ensured that you have the full chain available, so that you can do your upgrades, test them, and deploy them first when they're fully tested, instead of having to try to upgrade production machines without knowing if it'll work. It doesn't save you from untangling dependencies and version incompatibilities on upgrades, but it ensures that you can do that offline, and that when you're ready to upgrade, you have images you know works.

> Even if you have the source code the tools to compile it probably don't even compile on today's OS versions (if you can even get the right licenses).

You don't need to, as long as you retain the images of the versions the tools do run on. This is why I maintain most of my build environments in images too, so that I can retain not just the finished images, but the images used to build them.

> A docker with no coupling between the host and guest OS would make me happier. Then you have a binary you can just deploy 10 years after it has been compiled. And then you could extend this model to all kind of apps, including client/UI apps.

Sounds like you want a VM. Though Docker container come close in that you're depending on very narrow interfaces. I've got Docker containers running that were made from OpenVz containers that were built from machine images back in 2008. They're still running, if only because the original machine images were poorly documented enough that rebuilding them would have been a pain. But packaging them first for OpenVz and then for Docker worked just fine with minimal effort. I'm not sure what more you want.

> What it would really require is a new application model. With a simple interface with the OS to define all of its needs in term of firewall rules, ports to be opened, access to hardware, etc. And very few interactions beyond that

so you mean ... reinvent kubernetes?

No, he means ...provide a proper abstraction at the OS level (and across OSes even).

Not the clusterfuck that is kubernetes.

That's the good part of Docker. But it comes with the ugly part (tight coupling between the binaries and the version of the host OS).

What you would want is an interface that doesn't require that coupling.

More like Nixos services and modules.

He means SmartOS, just either doesn’t know it yet or shuns going out of his comfort zone.

Isn't SmartOS an hypervisor? Sure virtual machines achieve the right level of decoupling between the host and the guest but then you have to run your whole kernel, file systems, drivers, etc. You lose all the benefits of running an app/container.

POSIX is probably something closer to what I mean.

SmartOS is a hypervisor, an imaging system (via imgadm), a provisioning system (via imgadm / vmadm / Triton), a geographically distributed datacenter virtualization (via Triton) and an OS virtualizer (via Solaris zones). A SmartOS with a Joyent branded zone runs a native container at bare metal speed, with CPU bursting in the kernel as an added bonus, when the kernel detects a CPU intensive process. The whole thing is backed by ZFS and runs from memory. There is no installation.

As the Joyent branded zone is the native instantiation of the OS sharing the same kernel, there is no hardware virtualization, or rather, hardware is virtualized only once and zones then share the abstraction, as they are just normal UNIX processes. It's a completely different approach than hardware virtualization. One global zone can have up to 8192 non-global zones all sharing the same kernel, and the imgadm / vmadm combination makes the Docker problem become a non-issue.

SmartOS is an illumos distribution. illumos is a fork of OpenSolaris. Thus, SmartOS uses Solaris zones for what they call "OS-based virtualization" -- the systems all share a kernel and don't virtualize hardware. These are the same resource/performance benefits you get from Docker et al, and Zones facilities are built much more robustly.

Note that Solaris Zones predate Linux cgroups by several years, and that Zones offer full process isolation and were engineered explicitly for isolation/"OS virtualization" purposes, whereas cgroups were initially an attempt to give the kernel information about how to share resources between processes, and still do not offer real security separation or process isolation. cgroups is also a moving target, as "cgroups v2" or "unified cgroups" is still just barely becoming usable (CPU controller just added in 4.15, which is only a few months old).

You can run Docker images directly on SmartOS, because Joyent implemented the Docker API, and they also implemented a Linux compatibility API. It's not always perfect, but the fact that it works at all is admirable.

SmartOS also contains the equipment for hardware virtualization, and Joyent has even ported KVM (and recently bhyve) to illumos.

You can, theoretically, manage OS and hardware-virtualized systems at once through SmartOS, similar to what libvirt tries to provide on Linux (though their LXC integration is deprecated iirc).

I am not a SmartOS fanboy and have not as yet deployed it into anything more than a lab setting, but it, along with anything else non-Linux-centric, is consistently ignored for no real reason.

Should we do things in 2018 how we did them in 1998 just because it sort of worked in 1998? I say this as somebody who’s usually quite conservative in operations.

>Should we do things in 2018 how we did them in 1998 just because it sort of worked in 1998?

Well, if it ain't broken, don't fix it.

There’s a very fine line between painfully cumbersome and broken.

Just curious, do you still have/use a Nokia cell phone?

Funnily enough, I had to send my smartphone in for repair recently, and I popped my SIM card back in my old Nokia phone that I'd kept in a drawer -- it works great. It's a 10+ year old device and the battery has lasted all week.

Yes, helped me skip the whole "attention drainage", "mobile detox", "missing the concern while looking at your screen", "stolen privacy" thing...

No, but my old one still works, unlike my last 8 smart phones.

And still you use the smart phones daily, not the 'working' Nokia.

It's not binary. Robustness is something we lost. Yes, I am more happy (maybe? Perhaps I now just have another way to waste my time and be interrupted) with my smartphone, but I (we) lost something in the transition. That's the point; you're adding complexity and some people want to pretend that there's no trade off.

I agree, there is always a trade-off, and it is good to make it as explicit as possible.

Broken - no, tedious and resources consuming - hell yes

> because it sort of worked in 1998

Is "sort of worked" better than "only works until the next version of docker and docker's next new, new storage layer"?

If you need to add 100x the moving parts and hidden interactions to solve some hard problems then your hard problem just got harder. You've just kicked the can down to production.

I've had to work with lots of configuration management the past few years, and I'm always cleaning up the mess because we did this for 2 decades. The world may not have collapsed but I sure have lost weeks or months of productive time trying to untangle messy configuration management setups.