> small- or medium-sized businesses, which might have 50,000 VMs or fewer

Holy shit, is this considered small to medium enterprise now?

a single cluster supports approx 5,000 nodes, 110 pods per node~

The estimated maximum single cluster is 300,000 containers.

That's pretty medium, I've ran more than a million processes before, and nomad has 1million containers as its challenge https://www.hashicorp.com/c1m

borg can handle this easily.

I don't know anyone running 5k node K8S clusters. That said, borg (as an end user) appears to just keep scaling, but it uses a different model and makes different assumptions than K8S.

Sure. Most people have more clusters before they hit 5k nodes on a single cluster.

But I’ve been in situations where it would have been worthwhile. I’ve been in situations with 30,000 machines that needed to be controlled. Splitting them out into very many clusters would be a lot of wasted overhead in configuration, administration and because you lose nodes to masters.

I'm afraid this is why people pick Kubernetes. They believe small business needs those tens of thousands of VMs distributed across thousand of nodes and so on.

With some exceptions, I believe that's a few orders of magnitude above what a small business can run on. Nowadays people just start their day by drinking some K2l-aid and spinning up a "basic" 6-node cluster for a development prototype.

Maybe I'm wrong, of course.

> There are entire SaaS industries that could fit into a single Google/Facebook/Amazon datacenter.

Forget a whole datacenter, even just one rack is an unimaginable amount of computing power, these days!

Fact: An well-equipped Raspberry Pi 4 has more memory, more compute power, more storage and vastly faster networking than the Cray supercomputer I worked with at a major oil company in the 1990's! With the exception of the craziness around click-tracking (web ads and marketing have warped compute use even more than crypto), the data required to run even enterprise-scale businesses today is not really all that large.

For almost all purposes, we don't really need thousands of containers running on unimaginably fast computers, coordinated by AI-driven automation systems. What we need is software that is not morbidly obese.

I mean a single V100 GPU (~= 100 TFLOPS) has FLOPS throughputs similar to top end mid-00's supercomputers like Blue Gene, at least superficially. And you can squeeze like 4+ in a 1U if you have enough cooling and power.

https://en.wikipedia.org/wiki/History_of_supercomputing

Your scale (50,000 VMs) is way too high for small and medium sized business :-)

Is my observation correct that unicorns start to see that scale?

That’s a bit out of date. K8s can do 5,000 nodes and 300k VMs within its performance envelope: https://kubernetes.io/docs/setup/best-practices/cluster-larg...

A VM would be a node. A pod isn't a VM, it's a process tree.