I imagine this is because SQLite is VERY easy to bundle with Python itself. So on some platforms the OS SQLite is used, but on others it gets shipped as part of the Python installation itself.
It even works in WebAssembly via Pyodide!
Oh yes, I'm glad to see other with similar opinion.
Thanks for the great open source libraries yall!
It's kind of flaky and slow. It doesn't have namespacing. It's overly opinionated on the workflow (the way that states work, with a model version being in exactly one of dev, staging, prod is super hard to work with).
But beyond that, the biggest problem I have with MLFlow is what I call the "part of this complete breakfast" problem, which the ML/data-science arena is particularly susceptible to these days: the marketing talks a lot about what problems can be solved using the product, but not a lot about what parts of the problem the product actually solves. This is often because an honest answer to the latter question would be "not much". In the case of MLFlow, that would be totally fine, because honestly an opinionated CRUD app is a very useful thing. But it should be a lot more honest about what it does. It's not a system for automatically tracking model metrics, it's a database into which you can write model metrics with a known key structure.
- simple logging of (simple) metrics during and after training
- simple logging of all arguments the model was created with
- simple logging of a textual representation of the model
- simple logging of general architecture details (number of parameters, regularisation hyperparameters, learning rate, number of epochs etc.)
- and of course checkpoints
- simple archiving of the model (and relevant data)
and all that without much (coding) overhead and only using a shared filesystem (!) And with an easy notebook integration. MLflow just has way to many unnecessary features and is unreliable and complicated. When it doesn't work it's so frustrating, it's also quite often super slow. But I always end up creating something like MLflow when working on an architecture for a long time.
EDIT: having written this...I fell like trying to write my own simple library after finishing the paper. A few ideas have already accumulated in my notes that would make my life easier.
EDIT2: I actually remember trying to use SQLite to manage my models! But the server I worked on was locked down and going through the process to get somebody to install me SQLite was just not worth it. It's also was not available on the cluster for big experiments, where it would be even more work to get it, so I gave up on the idea of trying SQLite.