tknaup

With Kubernetes v1.24, non-expiring service account tokens are no longer auto-generated. This blog post highlights what this means in practice, and what to do if you rely on non-expiring service account tokens.

Co-founder of Mesosphere/D2iQ here.

Mesos started as a research project at Berkeley in 2009 and was originally focused on cluster computing frameworks like Hadoop. From the paper: "We present Mesos, a platform for sharing commodity clusters between multiple diverse cluster computing frameworks, such as Hadoop and MPI." It actually predates YARN by a few years. But, it very quickly (in 2010) saw production use at Twitter as the foundation for Twitter's custom PaaS which was later open sourced as Apache Aurora.

Marathon's main use case was actually for running microservice application in containers, which is why it has some advanced features around managing groups of containerized apps and their dependencies. The "meta-framework" use case for launching custom frameworks was also important but basically just needs Marathon to keep a container alive. Mesosphere never made Marathon proprietary. The full code is still OSS here: https://github.com/mesosphere/marathon/ Our commercial product DC/OS just added advanced workflows through a UI on top, and better integration with the rest of the components around Mesos.

ML is a relatively young field, and decades behind Software Engineering in terms of best practices for running production systems. CI/CD massively improved the innovation cycle time and quality of production software, and I believe it is key for building robust production ML systems as well. CML looks like a really easy to use product for bringing CI/CD to ML projects.

The new package is here: http://universe.dcos.io/#/package/beta-hdfs/version/latest

Mesosphere co-founder here.

You're correct in that GCP runs k8s in VMs, DC/OS doesn't. What's similar is that there's a resource manager underneath - Borg for GCP, Mesos for DC/OS. They serve similar purposes like resource management, isolation, and operating the services on top.

Mesosphere co-founder here.

This is correct: "DC/OS is better for running stateful services and then you can use K8 to run your stateless services"

Data services run directly on DC/OS via application-aware schedulers. They have the operational logic for how to bring up say a Cassandra cluster correctly, how to upgrade it to a new version without breaking it, change config, scale up, etc. All things you usually have to figure out yourself. When you run k8s on DC/OS you get these same benefits.

disclosure: I'm a founder of Mesosphere.

For most people k8s (like any other distributed system) is pretty hard to operate and wrap their head around. The goal of DC/OS is to make building and operating distributed systems easy. Mesos is pretty different from k8s. It doesn't have an API for microservice developers for example, instead it has low level primitives similar to an operating system kernel, on which you can build any software. We recently launched an SDK to make that even easier, along with a bunch of open source and commercial software that uses it, for example Cassandra/Datastax, Kafka/Confluent, Redis, Elastic, Couchbase, Alluxio. So DC/OS ends up looking a lot more like an entire cloud platform like GCP, which offers managed k8s, a managed database (BigTable), machine learning and so on. Under the hood it all runs on Google's proprietary Borg, which has some similarities with Mesos (Google is a sponsor of the lab where Mesos was created).

(Mesosphere co-founder here)

Running workload-specific schedulers like Kubernetes on Mesos is one of its fundamental ideas. From the paper:

"It seems clear that new cluster computing frameworks will continue to emerge, and that no framework will be optimal for all applications. Therefore, organizations will want to run multiple frameworks in the same cluster, picking the best one for each application."

https://people.csail.mit.edu/matei/papers/2011/nsdi_mesos.pd...

My favorite part is that moving Siri to Mesos actually had a noticeable impact on Siri's response time. It didn't just make the lives of Apple engineers easier but improved the product as well. For every one of the more than 700 million iPhone users worldwide.

self-serve machine learning and analytics is a really cool use case.

The new features are really exciting. Disk quota isolation is important to pave the way for stateful workloads, and with the upcoming work on stateful primitives (https://issues.apache.org/jira/browse/MESOS-2018) it will be easy to run things like databases and filesystems.

The service discovery metadata and labels allow more control over the visibility and naming of tasks. Mesos DNS (https://github.com/mesosphere/mesos-dns) will get support shortly.

Finally the modules also got some additions: hooks and anonymous modules, which can be used to add arbitrary extensions to Mesos.

Exciting release!