Containers _are_ a mechanism to increase to compute granularity. Some of our workloads were having trouble scaling to 128 cores, and by using containers, we can have more of them running with fewer cpu's. Furthermore, it's straightforward to support providing burst capacity to applications running within a container, given all the OS needs to do is to give those cgroups some extra CPU time for a limited time.

> Containers _are_ a mechanism to increase to compute granularity. Some of our workloads were having trouble scaling to 128 cores, and by using containers, we can have more of them running with fewer cpu's.

As opposed to having VMs? Otherwise, I don't see how containers can increase granularity.

As opposed to the original comment that says they aren’t useful for increasing granularity.

It sounds like containers enabled you to manage the deployment of multiple processes on a single machine more easily which doesn't contradict the comment you're replying to. Would it not have been possible without containers?

> containers aren't a mechanism to increase compute granularity

Yes they are. Instead of thinking in terms of a whole running operating system with dozens of services, now you are thinking in terms of individual (micro?)services that are relatively isolated from each other. We stuff a lot more containers per box than we used to stuff VMs per box.

But it's true containers (of the namespace/cgroup/seccomp variety) have failed to be a sufficiently secure isolation mechanism to use them for multi-tenant scenarios, so instead we mostly pack containers from the same owner together.

I'd sort of argue that Firecracker and gVisor are actually container engines that happen to use CPU features meant for hardware VMs for additional security hardening. The granularity of compute that you put in them is more container-ish than traditional VM-ish.

They are, or should be, entirely self contained such that whatever segregation is employed - be it hardware via a VM or in kernel with apparmor or SELinux provides sufficient segregation for the work load. V8s problem is JavaScript and NPM, but limiting the blast radius with hardware virtualisation is a win for segregation and v8 will win, at least for front end, because it’s got the mindset. As long as the library ecosystem cleans up.

"In kernel with apparmor or SELinux" can't possibly provide sufficient workload isolation, because it implies workloads share a kernel. It's easy to rattle off relatively recent kernel LPEs that no mandatory access control configuration would have prevented.

The Linux kernel simply wasn't designed to provide the kind of isolation "naive" containers want them to. Actually, generalize that out: Unix kernels in general weren't designed this way. It just doesn't work.

End game then is LittleKernel/Zircon on Fly.io? When do we get to play with those?

Whether software based access control is sufficient depends on the workload and where in the stack the workload runs. I agree though, hardware virtualisation based is more secure and less complex. It also requires access to bare metal, so a providers service or run it yourself, which is a trade off.

> over time, containers are all going to end up VM-scheduled, as VMs get cheaper and cheaper

That kind of started with Github actions.