Re:Mesosphere Announces First Data Center OS and $36M in Fun...
by presspot on Monday December 08, 2014 @09:32AM
(#8716561)
A bit of the history:
The Mesosphere DCOS is built around the Apache Mesos kernel
The Mesos kernel was developed at UC Berkeley in 2009 [1].
Spark was written as a sample app on top of it [2].
Ben Hindman and his colleagues at the UC Berkeley AmpLab had always envisioned Mesos as a kernel inside of a full-blown operating system [3]. They finally brought it to market.
[2] "We have implemented Mesos in 10,000 lines of C++. The system scales to 50,000 (emulated) nodes and uses ZooKeeper for fault tolerance. To evaluate Mesos, we have ported three cluster computing systems to run over it: Hadoop, MPI, and the Torque batch scheduler. To validate our hypothesis that specialized frameworks provide value over general ones, we have also built a new framework on top of Mesos called Spark, optimized for iterative jobs where a dataset is reused in many parallel operations, and shown that Spark can outperform Hadoop by 10x in iterative machine learning workloads." ibid.
The Mesosphere DCOS is built around the Apache Mesos kernel
The Mesos kernel was developed at UC Berkeley in 2009 [1].
Spark was written as a sample app on top of it [2].
Ben Hindman and his colleagues at the UC Berkeley AmpLab had always envisioned Mesos as a kernel inside of a full-blown operating system [3]. They finally brought it to market.
[1] https://www.usenix.org/legacy/event/nsdi11/tech/full_papers/...
[2] "We have implemented Mesos in 10,000 lines of C++. The system scales to 50,000 (emulated) nodes and uses ZooKeeper for fault tolerance. To evaluate Mesos, we have ported three cluster computing systems to run over it: Hadoop, MPI, and the Torque batch scheduler. To validate our hypothesis that specialized frameworks provide value over general ones, we have also built a new framework on top of Mesos called Spark, optimized for iterative jobs where a dataset is reused in many parallel operations, and shown that Spark can outperform Hadoop by 10x in iterative machine learning workloads." ibid.
[3] http://people.csail.mit.edu/matei/papers/2011/hotcloud_datac...