You can actually build an Omega vertical / Mesos framework architecture on Kubernetes, as described in this doc[1]. That doc pre-dated CRDs; the way you'd do it today is to build the application lifecycle management part of the framework using a CRD + controller, and run an application-specific scheduler (for pods created by that controller) alongside the default scheduler. The Kubernetes documentation page explaining how to run multiple/custom schedulers is here[2].

Borg only worked with a single scheduler, but Kubernetes allows you to build Omega/Mesos style verticals/frameworks and associated scheduling as user extensions to the control plane (as described above).

[1] https://github.com/kubernetes/community/blob/master/contribu...

[2] https://kubernetes.io/docs/tasks/administer-cluster/configur...

[Disclaimer: I work on Kubernetes/GKE at Google.]

> Borg only worked with a single scheduler

No love for rescheduler? =(

The rescheduler in Borg isn't a scheduler -- it just evicts pods, and then they go into the regular scheduler's pending queue and the regular scheduler decides where to schedule them. (At least that's how it worked at the time I left the project -- I assume it hasn't changed in this regard, but I don't know for sure.)

Because the name is confusing, we called the Kubernetes version of the Borg rescheduler the "descheduler" (https://github.com/kubernetes-incubator/descheduler) to make it clear that it doesn't actually schedule, just evicts. (There actually is something in Kubernetes called the "rescheduler" (https://kubernetes.io/docs/tasks/administer-cluster/guarante...) but it's a long story and we never should have named it that).