This is a bit disappointing and disconcerting to hear! I understand that technology doesn't happen in a vacuum, but when you have two well supported/known technologies, picking the most 'popular' one rather than something based on technical merit is a huge issue for me.
For example, the PetSet feature of Kubernetes 1.3 involved a really long discussion with people trying to work out how and why to use it.
I remember back in the Kubernetes 1.1 days when I was trying to get AWS ELBs to talk with Kubernetes and possibly finding drivers to automatically set up proxy points. In my search, I found the Github issues where people were discussing that very thing. The person who wanted AWS integration was there talking about the specific needs and quirks of the AWS ecosystem. There were a series of patches made to beef it up, and the discussion on Github documented how the feature was conceptualized (so as to be part of a more general solution on port management).
In contrast, I don't see that with Docker. I can't say anything about Mesos, as I've never tried using it.
Community goes a long way but has to co-exist with strong technology and being open to change.
Could you define the "everything" please.
What applications are you running? How many of them? How much resources do they use?
Since I have set-up both in production (and more than once) I can tell you for a fact. Kube is easier to setup and easier to run with less problems along the way.
Since I am running essentially the same cluster on kube that I ran on mesos, this is a no-bullshit comparison.
[To be fair] I never used dc/os, I used marathon and chronos on top of mesos.
I went to MesosCon this year and the previous. Last year it was a really fun kind of scrappy conference. This year it was basically "buy DC/OS". Even the open source vendor booths all had "works with DC/OS" stickers. It reminded me of "Dell recommends Microsoft Windows XP Professional for the best experience" crap that Microsoft used to pull with OEM's licensing windows. I was really unimpressed. Whereas Mesosphere is adding hooks into mesos to add things like AAA (Authentication, Auditing, Authorization) and then implementing it as proprietary modules in their commercial DC/OS, It is native and open source in Kubernetes (thanks to Redhat contributing code from openshift).
So on one hand I see a young ecosystem being swallowed by a single vendor who is slowly squashing the community. On the other hand I see several vendors doing everything in the open and a huge community. With the technical capabilities of both quickly converging as k8s implements a lot of the things Mesos used to have over it, Mesos is becoming less and less "sexy" and more like a liability unless you want to roll your own everything like Apple did with their Jarvis scheduler for Siri.
I am working on an open source project called "The Startup Stack" to essentially help startups bootstrap (and manage) a cluster in the cloud. link: http://docs.the-startup-stack.com/
Right now, it's working using mesos/marathon etc... 2 companies are using it in production.
In the last couple of months I've been thinking about making the move to Kube for this project. I might evaluate DC/OS as part of it.
Working on this project made me appreciate how hard it is to generalize and hit everyone's taste. There's a lot of customizing around cloud infrastructure now and I definitely feel there are more common grounds we can create.
I'll give DC/OS a shot before making my final decision.
Thank you for your suggestion and your awesome comments on this thread. I was always impressed with the mesosphere team in previous discussions here on HN.
Coincidentally, you announced the open source version on my birthday this year. Maybe it's a sign :)
Sorry I am new to this. I saw a presentation where someone was running Kubernetes on DC/OS . Why are people doing that if they can just run pure kubernetes.
for example, if you restart a Kafka node on any container orchestration platform, container is destroyed and respawn on another node which is great, but you lose all your data and the broker has to rebuild all that queue again, while Kafka on DC/OS wait and maintains the volume just in case you are having temporary maintenance restart before relocating the workload.
another example is Cassandra if you want to add more than 1 node, the traffic to resync the data across 2 nodes can actually degrade the performance on your cluster or even taking it down.
I can go on and on, all of these issues and requirements come from running these mission critical data apps in production, with each application having its own logic on how it handles day 2 operations, but you don't know what you don't know unless you tried running them in production.
Some people talk about the custom scheduler feature or Petsets in K8s, those features are cool and hats off to the K8s community, but they are still early in development while the framework architecture is a core mesos functionality since day 1.
And finally, everyone is free to run whatever they want/like, each company and person might have different preferences on how they want to run workloads and how much effort they want to go through that, I don't think it's productive to go to another VI vs emacs here ;-)
Kube is winning for the same reason React/Redux (and now Mobx) is winning and why Rails was winning at the time. Community.
The community for Kube is awesome and the work people are doing in the field is being noticed all over the place.
I've seen people (myself included) that moved production clusters from mesos to Kube just because the activity of the development and how secure they feel with the community and the project going forward.
React and Rails (at the time) had the same sort of pull towards the community and why a lot of people on-boarded.
Golang is most likely a factor here too. I feel most people find Golang friendlier than Scala/Java. That's why Kube has many more contributors, the hurdle for contributing is easier to jump