News for hackers. Stuff that matters.
Re:Torus: A Toolkit for Docker-First Data Science by sdeymanifold on Wednesday June 06, 2018 @06:21PM (#17251603)
Yes to containers! We are trying to make it as seamless as possible to be Docker first in all things. And not reinvent the devops wheel. It just needs to be adapted for the needs ot data scientists. Pachyderm is really cool. I will have to check it out. We've recently moved to Airflow for all our pipeline management... how does Pachyderm fit in that ecosystem?
Re:Torus: A Toolkit for Docker-First Data Science by jdoliner on Thursday June 07, 2018 @03:54AM (#17254060)
Pachyderm's pipeline system covers much of the same functionality as Airflow's so there's generally not much reason to use both.