(I work for GCP)

It looks something like this:

your container -> Compute Engine VM (GKE Node) -> container -> Borg

The container on top of Borg is used for scheduling and management. Joe's talk has a slide on this. As a GCP customer, you never have to worry about this or care about it, as it is an implementation detail.

>"The isolation is not perfect, though: containers cannot prevent interference in resources that the operating-system kernel doesn't manage, such as level 3 processor caches and memory bandwidth, and containers need to be supported by an additional security layer (such as virtual machines) to protect against the kinds of malicious actors found in the cloud."

As a GCP customer using GKE, your applications are separated from other GCP customer using VMs.

However, if you want to run your OWN untrusted workloads, then in the past you would have to spin up a separate VM for untrusted workload A and a one VM for untrusted workload B.

This sucks in terms of resource utilization. It would be better in many cases if you could run workload A and B on the same VM. That's where gVisor comes into play.

your untrusted container -> gVisor -> Compute Engine VM (GKE Node) -> container -> Borg

I hope this makes sense!

Thanks for the explanation, this makes sense yes.

>"The container on top of Borg is used for scheduling and management."

Is this the "open source node container manager" box on slide 13 then? I'm guessing this is the Borg's version of the kubelet then?

https://speakerdeck.com/jbeda/containers-at-scale?slide=13

That's a very old slide :) I "guess" the slide deck was talking about https://cloud.google.com/compute/docs/containers/deploying-c...

I see. So is "the container on top of Borg is used for scheduling and management" the Borg equivalent of the K8S kubelet then?

As you can see from the Borg paper [1] and the name, "borglet" is the most closest component to "kubelet".

[1] https://pdos.csail.mit.edu/6.824/papers/borg.pdf

(Co-author of the post)

The fact that gVisor is being used in multiple services at Google is probably the confusing part. In case of GKE Sandbox, the users here are external and using Cloud (specifically GKE). The target use case is to add defense in depth to their pods running on potentially shared GKE Nodes (VMs) for Multi-Tenancy. Our talk at Next'19 [1] includes a story by one of our customers, which may help understanding the use cases.

[1] https://www.youtube.com/watch?v=TQfc8OlB2sg

Thanks for the link that does make the use case clear i.e multitenancy/SaaS. Am I correct in assuming though that when someone creates a K8S cluster via GKE that the containers that make up their cluster such as the kubelets and masters are all running in VM underneath?

Yes, exactly. https://cloud.google.com/kubernetes-engine/docs/concepts/clu...