News for hackers. Stuff that matters.
Re:Show HN: Test, fix, and improve your ML models by esafak on Monday May 15, 2023 @07:58PM (#35956179)
I realize it is primitive today, but how is it eventually going to be different or better than aporia, arize, arthur, dioptra, evidently, gantry, graphsignal, guardrails, metaplane, superwise, uptrain, whylogs, etc. ?

Something for your site...

Re:Show HN: Test, fix, and improve your ML models by vikasnair on Monday May 15, 2023 @10:40PM (#35957218)
Comparison table is a great idea - we will add.

Some tools you mentioned here are monitoring solutions. Our value over them is that we catch issues in your models before those models ship. Monitoring is definitely essential, but to purely rely on it is to rely on your users to catch your bugs for you. Moreover, if you can find more issues ahead of time, you can circle back to retraining / recollecting data more immediately (and can dramatically bridge the dev cycle and iterate faster).

Beyond this, we also offer an extensive set of insights and tools for debugging that these other tools do not offer - including insights that use your model rather than the data (or just the predictions, as a proxy). Specifically, by uploading your model artifacts to Openlayer, you can see explainability (LIME and SHAP values) for your predictions and generate synthetic data to run counterfactual analysis on your model (to try and probe for edge cases not already represented in your data).

And finally, this is all baked onto a clean UX for tracking the versions of your models as you work towards improving the issues you find on Openlayer (almost like git for ML).