Re:Who needs MLflow when you have SQLite?
by nerdponx on Thursday November 17, 2022 @11:28AM
(#33641250)
Right, but experiments aren't always linear. Do you really want to make a new commit for every iteration of a hyperparameter search? What if you are using a black-box optimizer that supports parallel/concurrent updates?
I don't want to use Git to track all that. I want to use Git to store the final results of running such an experiment in the same commit as the code that implemented it. I just don't like the DVC experiment workflow, but I am more than happy to use DVC for storing the fitted model(s) at the end of the run.
Re:Who needs MLflow when you have SQLite?
by wxnx on Thursday November 17, 2022 @05:55PM
(#33647132)
Yeah, that's a fair point, and I agree. I don't think it's ideal.
Re:Who needs MLflow when you have SQLite?
by shcheklein on Thursday November 24, 2022 @01:50AM
(#33728446)
If you use `dvc exp run` you don't need to commit anything as I mentioned above. You can run multiple experiments in parallel, etc. Commit happens only / when you want to select the best result and share it with the team. But even that is optional.
I find the following workflow works well, for example:
1. Define steps depending on a `config.yml`.
2. Run an initial experiment (with an initial config) and commit the results.
3. Update config (preserving the alternate config and using symlinks from `config.yml` to various new configs if necessary), re-run, and commit.
4. Results are then all preserved in your git history.