Of course you do, you just moved the logic into the "orchestration" and "management" layer. You still need to write the code to correctly handle it. Throwing K8S at it is putting lipstick on a pig. It is still a pig.

> * We have a whole lot of HTTP services in a range of languages. Managing them all with fat binaries would be a chore - the author would have to give me a way to set port and listen address, and I have to keep track of every way to set a port. With a net namespace and clever iptables routing, docker can do that for me.

Nope, you wrote a set of rules and as long as everyone adheres to those rules things kind of work ( in a clever way ). Of course if you had the same kind of rules written and followed in any other method, you would arrive at the exactly the same place. In fact, you probably would arrive at a better place because you would stop thinking that your application works because of some clever namespace and iptables thing.

> * sometimes, I have to deploy and insecure app. Usually, it's a badly configured memcache or similar. With net namespaces, I can make sure only a certain server has access to that service, and that the service cannot ruin my host server.

You may be able to guarantee this with a VM but you certainly cannot guarantee it with a container.

> load balancing. Luckily, I don't have to deal with extreme network loads which would require hand-made solutions, but just pushing up that number a little to do ad-hoc load balance makes things a lot easier

The time to build a template and instrumentation for your haproxy/nginx/varnish layer is when you don't have enough traffic and enough complexities.

While a very large company can get away with building a custom in-house ecosystem that takes months for a new employee to come up to speed on, most companies find it more palatable to use industry standard products/systems that a new employee is already familiar with and can come up to speed on quickly.

Of course, but who would you rather hire, someone with solid fundamentals who can get up to speed quickly with anything, or someone who job-hops to use the latest buzzword library/framework/language that may have a useful lifetime of just a few years or even less? Given an either/or decision with other things being equal, I would almost always prefer to hire the former, and then use whatever tools worked best for the job regardless of whether they were built in-house or imported. A job-hopping resume full of trendy buzzwords is a big red flag for me.

While a very large company can get away with building a custom in-house ecosystem that takes months for a new employee to come up to speed on

This argument gets made all the time in discussions about frameworks, but I just don't see it. In my entire programming career, I have yet to encounter this hypothetical nightmare architecture that takes months before you can become productive and has no useful documentation or examples even though the entire application is built around it. (YMMV, obviously.)

Being able to get up to speed quickly on a new framework or library is an essential skill for a working professional developer. Whether that is the latest shiny Web World buzzword or something internal in your new employer's environment makes little difference, as long as the internal design is also sensible and reasonably documented -- and if it's not, you have much bigger problems than who you're thinking about hiring next.

Many highly competent individuals like to understand the fundamentals and first principles of how systems work and how different problems can be solved, and thus find work that provides no transferable knowledge/experience to be less appealing.

Of course, there is a balance here. Sometimes frustrating tasks must be done. In that case, ideally the end goal is sufficiently motivating.

Right, but to those people, whether you use Docker or Kubernetes or custom scripting or a unicorn whose horn magically creates new instances when you need them is mostly just an implementation detail. It's no more interesting than exactly which compiler or DB you use; these are just tools, means to an end. If the differences are significant for your use case, sure, you evaluate and choose accordingly, but then you use the tools to get on with the job. The deep, interesting stuff is almost always in what you can build once you're into that territory.

However, I do think it's likely there are some fundamental principles at play that make tool 1 better than tool 2 in specific contexts, and I think it is valuable and interesting to explore that.

Perhaps we are getting into the scientist vs. engineer mindset. I would guess, though, that the best engineers undoubtedly have some scientific curiousity helping them excel in their craft.

One extreme is that I see companies hold on to toxic employees because they're brilliant, even if they are bad for the company long term.

I can guarantee it pretty well on my RHEL and CentOS systems, SELinux prevents containers from accessing things they shouldn’t just like it prevents Apache from dumping /etc/passwd.

Slashdot tried that with CentOS and failed miserably. I have my doubts regarding your guarantee.

Don't group all container implementations into one category. You could certainly use FreeBSD Jails and Illumos Zones for isolating apps.

The most secure isolation is provided with physical separation. Second best is VMs. Containers are a distant third.

If I keep my boxes up to date I only need to worry about zero day kernel exploits. So if someone comes along and uses a zero day kernel exploit to escape my FreeBSD jail then they were going to get on the box anyway, because any attacker using a zero day is highly skilled and highly targeted.

> The most secure isolation is provided with physical separation. Second best is VMs. Containers are a distant third.

While I agree that physical isolation is the safest, I don't think it's so cut and dry between containers and VMs. Virtual machines have significantly more attack surface than either Jails or Zones, think of all the emulated devices.

Nothing prevents you from using jails or zones in your VM.

I get that but that's no longer VMs vs containers, Thats VM plus container vs container.

I will grant you that containers are harder to isolate since it's same problem as isolating individual processes. But there isn't really a meaningful difference between a kernel exploit and a hypervisor exploit.

Take a look at Clear Containers (despite the name, actually using VMs) and their qemu-lite minimal machine type.

Don't get me wrong I love people doing this. It makes my skills in uncomplicating needlessly complicated systems extremely valuable.

* Overhead. It might be getting smaller but it's certainly still there. Not just the application but all the OS services that are duplicated are extremely wasteful when you're billed for your resources directly.

* Your vendor distributes applications as docker containers.

* Your developers are building and testing in docker.

* Clustered resources are dead simple with swarm. It can obviously be done with VMs but it's work.

* Devs and sysadmins alike enjoy using off-the-shelf services that are maintained by someone else.

There are enterprise use cases where docker makes sense if the app support people are less impossible to deal with than the SAs. But you don’t have that issue in cloud.

Docker nowadays at least for me is essential for encapsulating st

* Not all our libraries are available in the same ecosystem. We need to mix JVM with C++ with a sprinkle of Python [via jep].

* We run a lot of small variations over the same code base. We'd rather not wait to re-deploy 99% of a fat binary for each run.

We have an old .net app we pushed to servers via zip files, and the sheer size of the zip ended up being a burden after many deployments. Docker's caching and "docker prune" make things a lot easier.

This is probably true, but it isn't good.

Because it still allows the base problem of crap coding practice to proliferate.

but nobody have to worry about it. just start a container the last guy left. /sarcasm

I don't think I believe that equality is nearly as neat you'd hope. Unless we're willing to call all code crap code, a lot of projects with well-respected code and development practices have had serious zero days, and we've also found that sandboxing software is highly effective. Insecure code in a secure, well-written sandbox can get you a secure deployment, and you probably want to have your "good" code in a well-written sandbox anyway.

bad code is often plaged with injection/sql injection. Which sandbox does an absolute NOTHING to protect you from. After all, it is all within the expected side effects of your code.

my point is that false sense of security makes it even easier for an attacker to use readily available privileges in the badly coded application.

* 'sufficiently' is doing a lot of work here. Go binaries might still need libc; Java tars-of-jars (or fat jars, if you're confused) still need a JVM, which needs a handful of shared libraries. But those things are relatively stable, generally backward-compatible, and one single big thing (libc or JVM), so it's fairly easy to manage having multiple versions installed anyway. More than zero work? Yes. Less work than managing a Kubernetes or Docker installation? Also yes.