News for hackers. Stuff that matters.
Re:ML Experiments Management with Git by icyfox on Thursday November 02, 2023 @06:20PM (#38121125)
I have a similar harness going for my recent experiments, except instead of hosting with huggingface I have a dataframe with pointers to the files on S3 and then just download them during local preprocessing.

Every time I see DVC mentioned I always feel like the idea was so close (and perhaps right in intuition to use git for everything) but the execution had just enough friction that I looked elsewhere. Small DX improvements really do cascade pretty far.

Re:ML Experiments Management with Git by nerdponx on Thursday November 02, 2023 @06:35PM (#38121317)
[delayed]
Re:ML Experiments Management with Git by unsynced on Thursday November 02, 2023 @06:50PM (#38121495)
FYI, you can use git worktrees [1] to work on multiple branches simultaneously

[1] https://git-scm.com/docs/git-worktree

Re:ML Experiments Management with Git by nerdponx on Thursday November 02, 2023 @08:31PM (#38122593)
Yeah, I know and love that feature for software projects, especially if I need to switch over to a bugfix while I'm deep in a topic branch.

But for a data project it would be a big pain to have separate worktrees just to work around what IMO is a usage anti-pattern to begin with!

Re:ML Experiments Management with Git by shcheklein on Thursday November 02, 2023 @10:51PM (#38123851)
DVC has `dvc exp` that doesn't require creating commits or branches. It's utilizing git custom references (technical details [1]). And it can be visualized in CLI or VS Code.

[1] https://iterative.ai/blog/experiment-refs

[2] https://marketplace.visualstudio.com/items?itemName=Iterativ...

Re:ML Experiments Management with Git by nerdponx on Friday November 03, 2023 @05:35PM (#38135289)
[delayed]
Re:ML Experiments Management with Git by a_bonobo on Friday November 03, 2023 @02:53AM (#38125370)
I've really liked the idea of scidataflow in this context: https://github.com/vsbuffalo/scidataflow

It's neat for research as it stores the data on scientific data repositories like Zenodo and you get DOIs.