Log and track ML metrics, parameters, models with Git and DVC

Re:Log and track ML metrics, parameters, models with Git and... by londogard on Sunday May 21, 2023 @04:13AM (#36018897)

I really enjoy using DVC. I do have some drawbacks compared to other offering like MLFlow and W&B.

1. Harder to track experiments on remote VM's (e.g. Azure) as there's no server (we need to feed results back somehow) 2. Impossible (?) to track different types of experiments in the same repo. MLFlow has a way to define experiments and runs, which means I can easily group Regression vs Classification or even if I try a completely different task with the same data.

If anyone has a good suggestion on how to solve these two I'd love to fully commit to DVC!

[ Parent | on HN ]

Re:Log and track ML metrics, parameters, models with Git and... by henrydark on Sunday May 21, 2023 @11:41AM (#36022057)

1. I git commit with a token

2. Master has the deployed models, other experiments are in branches - just like other experimental code

[ Parent | on HN ]

Re:Log and track ML metrics, parameters, models with Git and... by londogard on Tuesday May 23, 2023 @07:32AM (#36042820)

Thanks!

We solved 1. the same way, but it felt "off" somehow. Perhaps it's a good solution.

2. That's a sound solution, but a tiny bit cumbersome. I have projects where we deploy both classifier and regressor, where it'd nice to keep all in main. Alas, you can't have it all.

[ Parent | on HN ]