News for hackers. Stuff that matters.
Re:Databricks open-sources Delta Lake to make data lakes mor... by jdoliner on Wednesday April 24, 2019 @04:33PM (#19742321)
Throwing my own project's hat in the ring, Pachyderm[0] is opensource, written in Go and built on Docker and Kubernetes. It versions controls your data, makes modifications atomic and tracks data lineage. You implement your pipelines in containers, so any tool you can put in a container can be used (and horizontally scaled) on Pachyderm.

[0] https://github.com/pachyderm/pachyderm

There are no comments on this comment yet.