That said, Google, to my understanding, does run a completely containerized infrastructure internally, including databases and other stateful things, so it is not wildly off to suggest running a database on Kubernetes.

Before it ran on F1/Spanner, Adwords ran on sharded MySQL on Borg. Not for its entire early life, but for quite a few years. Later in life, Checkout and maybe Wallet ran on MySQL on Borg, too. So did YouTube, which used Vitess, a sharding layer (now ported to Kubernetes and open sourced).

Bigtable, Spanner and even Colossus/D run in containers on Borg.

How is colossus running in a container in borg? That's their network file system that makes running databases in containers possible?

There's a bootstrapping issue, of course, but that's how it works.

And it's even crazier than what you're picturing. What if I told you that Bigtable runs on top of Colossus, but Colossus itself stores metadata about files, including Bigtable's files, in... Bigtable? It's really turtles all the way down, the last of which is, luckily, Chubby.

Some of the details here: http://www.pdsw.org/pdsw-discs17/slides/PDSW-DISCS-Google-Ke...

Packaging was (is?) a bit different, because it existed before Docker existed.

Pdf of the presentation: https://www.usenix.org/sites/default/files/conference/protec...

And a recording of the talk: https://www.usenix.org/conference/lisa14/conference-program/...

Googlers, including the ones working on Kubernetes, like to pick on MPM, but that's probably because they haven't run major services deployed from Docker images.

Kubernetes might be simpler than Borg in so many ways (let me count the ones I care about...), but it also has better features that Borg did not implement (labels and selectors) or that are offered only by some other internal services, which obviously are configured through very different mechanisms (ingress).

Wow, I'd never heard of Fargate until this. It seems so somewhat obvious now for things that Lambda doesn't fit. Heroku for containers, not VMs.

There's also Azure Container Instances: https://azure.microsoft.com/en-us/services/container-instanc...

And Google has AppEngine, now with Flexible Environment that can run containers: https://cloud.google.com/appengine/docs/flexible/dotnet/quic...