VM overhead isn't trivial. It still remains a pretty big factor in terms of cost bloat for CPU-bound stuff. Also, VMs take a godawful long time to start up; if you care about, say, responding to load within ten seconds, VMs aren't a great choice.

They're fine for a lot of things, of course. I use them all the time. But I use containers for other things.

I recall several reliable testers confirming that the CPU overhead of virtualisation was negligible, somewhere around 2%. Unfortunately I could not quickly find those papers now, but I did find a old VMWare whitepaper[1] showing they had ~7% overheard 5+ years ago, which sounds about right considering what kind of advancements they would have made in half a decade.

[1] http://www.vmware.com/pdf/hypervisor_performance.pdf

Sounds feasible, but CPU usage isn't really talked about as an advantage of containers.

I expect startup time and memory usage would be lower, but to my mind the advantages are mainly around flexibility... e.g. How long it takes to create or upload an image file. How long it takes to set up a minimal infrastructure with several components to it on a single EC2 instance. Decoupling the operating system patch cycle from the app deployment image generation cycle. etc.

It's just MUCH more memroy efficient to run containers and also VMs typically have worse I/O throughput. CPU scores are fine though.

As an example i am running around 20 containerized servers on my Laptop in a 4GB VM which would typically be run on 20 distinct VMs on one or more hypervisors. It's not very fast but the density of servers you can put on your hardware is MUCH bigger.

"if you care about, say, responding to load within ten seconds, VMs aren't a great choice."

That's actually exactly why I would use a VM..

You can spin up, configure, and push into production an application in a new virtual machine in ten seconds? I'd like to see proof of that.

The best I've managed was ninety seconds on my own hardware and three minutes (on average) in AWS.

Ah sorry! I didn't think you meant literally "10 seconds", was assuming you just meant quickly (a few minutes).

I can't really think of a use case though where someone would need more capacity in sub 10 seconds. Maybe if you only intend to scale horizontally with a bunch of 500Mb instances and had little to no room to set an appropriate scaling threshold? What would be a couple examples? With the apps I've seen the past several years generally they have scaling thresholds at 'X' resource and 3 minutes is more than enough to provision extra capacity for their needs.

Also, kind of ironic but your site is giving me a 503 :p

Just lighter weight. Ways that containers can be cool:

You need to start 10 containers locally to dev your app. Spinning up a local 10 VMs kind of sucks. 10 containers can be pretty quick.

(now if you really need 10 containers is another question, and some people clearly over split their architecture).

Containers are just a way to launch threads without polluting your local namespace or system. It's a way to say "hey, this stuff shouldn't interfere with anything else".