Re:Log and track ML metrics, parameters, models with Git and...
by londogard on Sunday May 21, 2023 @04:13AM
(#36018897)
I really enjoy using DVC. I do have some drawbacks compared to other offering like MLFlow and W&B.
1. Harder to track experiments on remote VM's (e.g. Azure) as there's no server (we need to feed results back somehow)
2. Impossible (?) to track different types of experiments in the same repo. MLFlow has a way to define experiments and runs, which means I can easily group Regression vs Classification or even if I try a completely different task with the same data.
If anyone has a good suggestion on how to solve these two I'd love to fully commit to DVC!
Re:Log and track ML metrics, parameters, models with Git and...
by londogard on Tuesday May 23, 2023 @07:32AM
(#36042820)
Thanks!
We solved 1. the same way, but it felt "off" somehow. Perhaps it's a good solution.
2. That's a sound solution, but a tiny bit cumbersome. I have projects where we deploy both classifier and regressor, where it'd nice to keep all in main. Alas, you can't have it all.
1. Harder to track experiments on remote VM's (e.g. Azure) as there's no server (we need to feed results back somehow) 2. Impossible (?) to track different types of experiments in the same repo. MLFlow has a way to define experiments and runs, which means I can easily group Regression vs Classification or even if I try a completely different task with the same data.
If anyone has a good suggestion on how to solve these two I'd love to fully commit to DVC!
2. Master has the deployed models, other experiments are in branches - just like other experimental code
We solved 1. the same way, but it felt "off" somehow. Perhaps it's a good solution.
2. That's a sound solution, but a tiny bit cumbersome. I have projects where we deploy both classifier and regressor, where it'd nice to keep all in main. Alas, you can't have it all.