yoshiat

"This guide provides best practices to safely and efficiently set up multiple multi-tenant clusters for an enterprise organization."

This "multiple" multi-tenant clusters part isn't coming through. Please do jump into "Securing the cluster" section to cut corners and learn what to do in a single cluster. We're fixing the sections to avoid the confusions. Thanks for the feedback!

https://cloud.google.com/kubernetes-engine/docs/best-practic...

Sorry if it was confusing. You need to read into more about how to set up in-cluster multi-tenancy.

We do recommend robust configurations for production setup (e.g. dev, staging and production) however you can certainly squash and skip it if not necessary.

Thanks for the feedback though. We'll consider adding such notes explicitly.

For secure isolation, we learned it's not sufficient. It's good for resource isolation though.

PTAL at https://www.youtube.com/watch?v=6rMGRvcjvKc

For robust configuration yes. However one can certainly collapse/shrink if having multiple clusters is going to be a burden cost-wise and operation-wise. This best practices was modeled based on the most robust architecture.

Exactly!

https://cloud.google.com/kubernetes-engine/docs/best-practic...

Thanks! We actually do and shipped together with the best practices.

https://github.com/GoogleCloudPlatform/gke-enterprise-mt

Please give us feedback there in case you hit any issue!

PTAL at doing Multi-Tenancy in GKE!

https://cloud.google.com/kubernetes-engine/docs/best-practic...

We don't recommend using node pools for isolation.

Yes and it's going GA.

https://cloud.google.com/blog/products/containers-kubernetes...

https://cloud.google.com/kubernetes-engine/docs/best-practic...

Nit: we don't recommend dedicated nodes for isolation. PTAL at https://www.youtube.com/watch?v=6rMGRvcjvKc And the guidance from GKE is at https://cloud.google.com/kubernetes-engine/docs/best-practic...

thockingoog's responses below articulates my feeling very well.

https://news.ycombinator.com/item?id=22487726 https://news.ycombinator.com/item?id=22487110

What I can do is to lower the bar for aggregating clusters with the investments we've done so far. Hope that makes sense.

Yes, namespace alone isn't sufficient for isolation. Would you be able to look at our latest Multi-Tenancy best practices?

https://cloud.google.com/kubernetes-engine/docs/best-practic...

It's a living product which comes with Terraform modules. We introduced various features to enable doing Multi-Tenancy as well (and more on their way!)

Thank you for your feedback and we understand this was a surprise to you and many.

For cluster per customer architecture, would you be able to look into https://cloud.google.com/kubernetes-engine/docs/best-practic... to see if there is anything useful? We understand changing the architecture isn't easy at all and we'd love to know how we can help.

As you can see from the Borg paper [1] and the name, "borglet" is the most closest component to "kubelet".

[1] https://pdos.csail.mit.edu/6.824/papers/borg.pdf

That's a very old slide :) I "guess" the slide deck was talking about https://cloud.google.com/compute/docs/containers/deploying-c...

Yes, exactly. https://cloud.google.com/kubernetes-engine/docs/concepts/clu...

I replied in other comments but our talk at Next'19 [1] includes a story by one of our customers, which may help understand the use cases. In a nutshell, GKE Sandbox should allow sharing the resources of GKE Nodes (VMs) among multiple tenants.

[1] https://www.youtube.com/watch?v=TQfc8OlB2sg

(Co-author of the post)

The fact that gVisor is being used in multiple services at Google is probably the confusing part. In case of GKE Sandbox, the users here are external and using Cloud (specifically GKE). The target use case is to add defense in depth to their pods running on potentially shared GKE Nodes (VMs) for Multi-Tenancy. Our talk at Next'19 [1] includes a story by one of our customers, which may help understanding the use cases.

[1] https://www.youtube.com/watch?v=TQfc8OlB2sg