We rebuild ours all the time. New config, k8s version upgrade, node OS patching.

Interesting. I'm only familiar with Mesos/Aurora, which is often considered outdated next to Kubernetes, but it can do all those things in place.

Do you end up with a "meta-kubernetes" to deploy kubernetes clusters and migrate services between them?

You definitely can do the same with Kubernetes too, just that the scope is too large and it doesn't have a good reputation with rolling updates of controlplane.

> Do you end up with a "meta-kubernetes" to deploy kubernetes clusters and migrate services between them?

Congratulations, you just discovered the ClusterAPI

The management cluster is definitely a pattern. Cluster API is designed to have a management cluster to build workload clusters across region and provider.

https://cluster-api.sigs.k8s.io/

People work in different ways. We do in place upgrades across all components. In fact that's sort of why we use k8s in the first place.

Also, I have an allergy to complexity, so I need fewer moving parts for health reasons.

Most of the times it is because of fear of breaking something during a rolling update. So people usually create a new cluster from scratch and slowly migrate one by one.