ferrantim

This is super interesting. Sharing with my kids who are starting geography in school this year.

Gonna have to try this. Been using bettersnaptool for years.

Different topic but this article got me lost down a rabbit hole looking for something similar for the TI86. Ah, memories...

Just wanted to say thanks for the article. Very well-written and actionable.

Yes please. Blog would be awesome.

Portworx employee here. Excited about joining Pure! Great company, amazing support and very product- and innovation driven.

I can attest that persistent storage is the hard part! Full disclosure, I work for a company[1] who makes a persistent storage solution for containers/Kubernetes. We are absolutely seeing that our large customers (folks like GE, Verizon, Dreamworks, Comcast, etc) are running "cloud native" applications on-prem as well as in the public cloud so this is a really smart move for Google.

[1] https://portworx.com

NIO, the self-driving car company is doing this. They did a pretty detailed interview on their use case which includes a 120 PB data lake, and Cassandra, Kafka, TensorFlow, HDFS. You can read here: https://portworx.com/architects-corner-kubernetes-satya-koma... . (Disclosure, I work for Portworx the solution they use for container storage, but hopefully the content speaks for itself).

Congrats Luke and team. I'm curious, what did you learn at ClusterHQ with Flocker that made you want to start dotmesh?

Michael from Portworx here. Thanks for the shout out. Just for some context, we just announced a partnership with Mesosphere today to help accelerate adoption of DCOS for stateful services [0] in fact. We handle the automation of all the state management mentioned above, not just volume provisioning. Our customers include big companies like GE and Dreamworks but also a lot of smaller companies. You can use PX-Dev[1] for free up to 3 nodes. Would love feedback.

[0] http://m.marketwired.com/press-release/mesosphere-partners-w...

[1] https://docs.portworx.com

I'm biased but checkout Portworx. It is designed to be that and if you don't think it is, I'd be curious why is missing. https://portworx.com

You could look at Portworx (disclosure, I work there so am biased, but you can test it yourself for free)

Correct, it is not opensource but you can use it for free on up to 3 nodes. Not sure what "enterprisey" means but to me that sounds like it works well for large deployments and that is true. We have customers running us with Mesos on up to 1000 nodes and K8s on hundreds of nodes.

Ceph is not a good solution at all for databases. Your performance is going to be terrible.

Ceph is object-based SDS solution designed to take servers with local drives and create a SAN out of them. In order to do this, they take each LUN (Ceph volume) and scatter the data across all nodes in the cluster. They do not assume that applications will run on these servers themselves... they assume compute is elsewhere, like a traditional SAN. The goal is to replace a SAN with servers, not create a converged platform. Also, Ceph was designed during an age where an Intel server did NOT have tier-1 capacity (8 - 20 TB), which is why they shard a volume across so many servers.

This causes a problem for modern applications like Cassandra, Mongo, Kafka etc, where they like to scaleout themselves and want a converged system, where data is not scattered, but on the node where an instance of that cluster runs. Ceph also disrupts (undo) the HA capabilities that these scaleout applications have (For example, a Cassandra instances data will not be on a node on which it thinks it is).

You can use Portworx[1] to transfer data between servers and datacenters. It has a native kubernetes driver, is production ready being used by GE and Lufthansa airlines among others and does sync and async replication of data between environments. (disclosure I work for Portworx). [1] https://portworx.com/

Do msft, amazon or clarifai support video like Google just announced last week?

Congrats to the entire Mailgun team. I'm looking forward to seeing all the awesome stuff you do an an independent company!