kozyraki

The breakdown for the $1.7M toilet was published: $300K for the architect, $125K for construction management, $175K for the City project manager, etc. All this for one toilet in this cute town square. This has nothing to do with humanity, fascism, or the city being full. It's simply gross public $ mismanagement. SF should use this $ for something more useful, like better schools.

Or we can automate this in an OS/cluster manager and safely this with everyone like we do with some many other technologies that are difficult for most of us to develop :)

I learn something every day :)

All of us who have used Borg over the years are very appreciative of the technology and its capabilities. Congrats to all the Googlers, current and ex, that contributed to it. This paper should be toward the top of the reading list for anyone working on the topic.

Nevertheless, there are many open questions for large-scale cluster management for researchers and developer to address. Here are some of my favorite: - The curse of overprovisioning: Borg and many other systems rely on reservation which are systematically exaggerated by users. Right sizing these reservations is one way to go beyond the 40-50% usage shown in the Borg paper (see fig 12). A promising way of doing this is Christina Delimitrou's work using classification techniques (see http://goo.gl/vFf8oN) - Oversubscription using better isolation mechanisms): this is what the Borg paper calls resource reclamation. Take unused (but reserved) resources from priority jobs and use them for best effort analytics. David Lo (http://web.stanford.edu/~davidlo/) has a very interesting paper coming up on how to coordinate cpu sets, cache partitioning, Linux TC, RAPL/DVFS (power management) to run websearch clusters at >90% by packing them with analytics without causing ANY glitch on search. And that is Google search.

There are definitely more interesting. Exciting times.

A quick summary of what Mesosphere is working (DCOS - the datacenter opearting system) on is at https://mesosphere.com/ .

Apache Mesos, the core of Mesosphere's infrastructure, has been in use by companies like Twitter for years to manage clusters with 10s of thousands of servers. http://mesos.apache.org/

Thanks for the comments everybody. I am one of the people behind Mesos-DNS and will gladly answer any questions about it.

DNS is an obvious way to support many service discovery needs. There were two motivations for implementing Mesos-DNS instead of using another DNS system. First, we needed a DNS system that closely integrates with Mesos. Instead of having every user or framework describe tasks twice (once to Mesos for execution and once to a DNS system), it easier and cleaner to automatically pass task information from Mesos to DNS. Second, we wanted a simple solution. Mesos and its frameworks already implement fault tolerance and life cycle management. We did not want to force Mesos users to deploy another set of consensus mechanisms, persistent storage, or a replicated logs just for DNS.

One more interesting detail. SkyDNS, Consul, and Mesos-DNS use the great go-DNS library by Miek Gieben (github.com/miekg/dns). So, their core DNS capabilities are the same. They differ in what they offer for fault-tolerance, life cycle management, load balancing, etc. Mesos-DNS defers all this functionality to Mesos and its framework (KISS).

Sure, pthreads and other thread libraries that include a lot of runtime functionality would probably be a better analogy.

My view on this the following: Mesos is similar to the kernel of a conventional operating system (e.g. Linux). It provides very basic services (scheduling, interrupts, device management, etc) and a syscall API. But nobody wants to program to this API. Hence you need libc or other similar libraries to provide a higher level API that programmers use to interact with the kernel. Kubernetes, Marathon, Aurora, etc are such libraries, each optimizing for a different class of applications and providing different functionality. The two (the kernel and the libraries) need each other.

Dune and its follow-up (IX, see http://csl.stanford.edu/~christos/publications/2014.ix.osdi....) can be used to secure and accelerate containers using Intel virtualization hardware. It'd be great if the community puts some energy behind these ideas.