Re:Show HN: Quilt – manage data like code
by lobster_johnson on Friday July 14, 2017 @03:31PM
(#14772512)
Pachyderm was the first thing I thought of. In fact there should be opportunities for collaboration here.
Pachyderm looks brilliant for high-scalability parallel data processing, and the versioned data part is a way to not just maintain the history of the data, but also avoid reprocessing of data that hasn't changed since the previous run.
Re:Show HN: Quilt – manage data like code
by jaz46 on Friday July 14, 2017 @06:17PM
(#14773658)
Hi, I'm one of the creators of Pachyderm. We've been talking with the Quilt founders about various ways to work together and think there are some really exciting opportunities!
Pachyderm looks brilliant for high-scalability parallel data processing, and the versioned data part is a way to not just maintain the history of the data, but also avoid reprocessing of data that hasn't changed since the previous run.