Can you elaborate why they are correct with this view? What is inherently superior about 1 process per container? What is wrong with having two cooperating processes per container? And why should Docker containers only run on bare metal?
The "point" of Docker is to be a lightweight replacement for virtualization in some cases. Layering Docker on top of virtualization, while fine, adds layers rather than removing/replacing them.
As for running on bare metal, I don't think the implication is they always better, but that if your goal is huge scale minimizing the number of layers is better for efficiency (though not necessarily better for other things - e.g. you may want to group/sandbox some containers in VMs for increased security)
At sufficient scale, horizontally scaling is complicated and involves implementing the things I avoid until I have no choice [Dynamic Service Discovery on a per-process basis, etc.]. I'm mostly deferring to stuff I've read about it since I haven't really worked at companies that are larger than $XX million a year.
It generally is things like principle of least privilege, the fact you already have to bite the bullet on managing your service discovery through yet another tool, etc.
Personally, I've never dealt with anything larger than 2-3 datacenter locations with a rack of equipment in each.
> And why should Docker containers only run on bare metal? That you can just ignore.
I really just meant don't do:
bare metal -> Spin up Xen VM -> Spin up docker containers
Any of these are valid imo:
AWS -> Docker Containers
Bare metal -> Docker Containers
[Insert hosting option X] -> Docker Containers
The maintainers of Docker appear to strongly believe that 1 application per container is the way to go.
At sufficient scale, they are pretty much correct. You grab Y bare metal hosts and spin up X docker containers for X processes.
> Isn't this just pushing the problems into managing dependencies between more little VMs while also constraining the architecture? It increases the difficulty of synchronizing 10 or 20 separate isolated applications start,upgrade,fail. Maybe all that living on yet another VM guest machine (so conceptually having 2 virtualization levels). Handling more complicated network setup (bridging, firewall rules at multiple levels). Handling effects of disk and other subsystem interacting with each other in strange, sometimes sub-optimal ways.
A handful of people advocate for immutable role-based containers [e.g. https://devopsu.com/blog/docker-misconceptions/] for that reason. In that use case, its really replacing running something like Xen + Chef, KVM + Ansible, or whatever.
You grab a host machine, you stand up your X containers with Z processes per container on Y hosts.
I think this is really equivalent to the Microservices vs. Normal SOA vs. Monolithic argument. Given sufficient scale/requirements, each makes sense. However, none of them are optimal for all situations.