So even if the scheduler is vaguely resource-aware (I'm not convinced that's true) it would be entirely static, based on things like container count.
To be more concrete: Within the PodSpec type that you linked to, there is a field of type []Container. Within the Container type there is a field called Resources which is of type ResourceRequirements. ResourceRequirements lets you specify resource requirements of the container. The resource requirements of the Pod are computed by adding up the resource requirements of the containers that run within the Pod.
In addition to resource-based scheduling, we also support "label selectors" which allows you to label nodes with key/value pairs and then say that a Pod should only run on nodes with particular labels. That's specified in the NodeSelector field of the PodSpec (which you linked to).
The Kubernetes scheduler looks at "fit" and "resource availability" to determine the node a pod will run on. Nodes can also have labels like "high-mem" or "ssd", so you can request a particular type of server (via the the nodeSelector field). More details are in the link above.
The documentation on resource-based scheduling is at https://github.com/GoogleCloudPlatform/kubernetes/blob/maste...
The new documentation you linked to has good explanations in it as well.
DCOS is some really nice packaging for marathon, chronos, etc, with a nice cli tool for downloading and installing new frameworks onto mesos. Personally, I find using Aurora a lot nicer.
That being said, I do think the kubernetes-mesos gives you far and above the best of both worlds. You get the developer story of kubernetes, with the ops story of mesos.
From an ops perspective, k8s is a bit clunky. I was actually shocked when I found out after bringing a new kubelet (worker node) online, you have to also update a service on the master. This was in a power training class on Kubernetes at this year's Redhat Summit in Boston. Really underwhelmed with the complexity of k8s compared to mesos, but they aren't an apples to apples comparison.
On AWS that has not been my experience. Nodes can be brought up dynamically and they register themselves with the master.
kubectl get minions
Or usable at all. Really frustrating to me.
- Mesos is a generalized, low level framework for distributing workloads over multiple nodes. It provides mechanism not policy. Therefore it requires quite a bit of up-front work to build something usable for any given application.
- Kubernetes is an opinionated cluster execution tool. It provides tools and a curated workflow for running distributed containerized applications. It's generally pretty quick and easy to get running.
- Mesos has a rich, resource-aware task scheduler. You can specify that your application requires X CPU units and Y RAM units and it will find the optimum node to run the task on.
- By contrast, the Kubernetes scheduler currently is rather dumb[1]. There's no way to specify the expected resource utilization for pods, and the scheduler simply tries to spread out replicas as much as possible throughout the available nodes.
People are (rightly) excited about things like Mesosphere which could allow the best of both worlds: the ease and API of Kubernetes with a powerful Mesos resource scheduler, not to mention nice-to-haves like a Web UI with pretty visualizations.
You can now cut me a check for 50% of the consulting revenue you get from this information. :)
1. The scheduler is intentionally simple and pluggable, to allow improvements easily in the future. My statements only apply to the current state of Kubernetes as deployed today.