News for hackers. Stuff that matters.

mcncfie

Thanks! Regarding 2, could you give an example?

Also, can I combine DVC with a pipeline tool like Apache Airflow?

Hmm. If I understand correctly, in order to reproduce the steps taken in creating machine learning models, I need to version control more things than just the code:

1. Code

2. Configuration (libraries etc)

3. Input/training data

1 and 2 are easily solved with Git and Docker respectively, although you would need some tooling to keep track of the various versions in a given run. 3 doesn't quite figure.

According to the site DVC uses object storage to store input data but that leads to a few questions:

1. Why wouldn't I just use Docker and Git + Git LFS to do all of this? Is DVC just a wrapper for these tools?

2. Why wouldn't I just version control the query that created the data along with the code that creates the model?

3. What if I'm working on a large file and make a one byte change? I've never come across an object store that can send a diff, so surely you'd need to retransmit the whole file?

Re:Grafana 4.0 with alerting is released on Tuesday November 29, 2016 @07:08PM (#13067354)
Congrats guys!
Re:ArangoDB Closes 2.2M Euro Investment Led by Target Partners on Thursday November 24, 2016 @12:36PM (#13031527)
I have a lot of faith in multi-model datastores but I'm a little worried about the Jepsen test post below. Is that true?