News for hackers. Stuff that matters.
Re:We Need DevOps for ML Data by jdoliner on Tuesday April 28, 2020 @05:55PM (#23012614)
Pachyderm is probably one of the companies you've seen in this space. Full disclosure: I'm the founder, but I feel that we've stayed pretty true to the idea of being a modular open-source tool. We have customers who just use our filesystem, and customers who just use our pipeline system, and of course many more who use both. We've also integrated best in class open-source projects, for example Kubeflow's TFJob is now the standard way of doing Tensorflow training on Pachyderm, and we're working on integrating Seldon as the serving component. We find this architecture a lot more appealing than an all-in-one web interface that you load your data into.
Re:We Need DevOps for ML Data by gas9S9zw3P9c on Tuesday April 28, 2020 @06:05PM (#23012703)
I haven't used you yet, but IMO this is the way it should be done. Once I get around to cleaning up my current custom k8s pipelines I will give you a spin :)