Re:Why software engineering processes and tools don’t work f...
by kthejoker2 on Thursday October 15, 2020 @02:43AM
(#24785570)
It's sad to see such "to hell with it" naivete (and the comments here with broad supporting evidence that data scientists are poor software developers.
There are also a lot of DS people wrestling with the deeper challenges at the heart of MLOps and beyond just advocating sane team-based workflows, are looking at ways to harden and structure ML specific processes in the planning, design, build, and deploy phases.
Test coverage of ML models and a good discussion on why and how ML code testing isn't the same as traditional testing - https://www.jeremyjordan.me/testing-ml/
And of course a whole suite of tools and research beingn done on data, pipeline, and model management, optimizing deployment and serving, explainability and interpretability, all in the service of applying enterprise-ready, engineering-grade rigor to data science.
And then you read something like this and want to tear your hair out.
There are also a lot of DS people wrestling with the deeper challenges at the heart of MLOps and beyond just advocating sane team-based workflows, are looking at ways to harden and structure ML specific processes in the planning, design, build, and deploy phases.
Sean Taylor at Lyft on choosing metrics - https://medium.com/@seanjtaylor/designing-and-evaluating-met...
Test coverage of ML models and a good discussion on why and how ML code testing isn't the same as traditional testing - https://www.jeremyjordan.me/testing-ml/
A tool for Behavioral testing of NLP models - https://github.com/marcotcr/checklist
How to be a 10x data scientist (spoiler: have a professional end to end workflow) - https://towardsdatascience.com/how-to-be-a-10x-data-scientis...
Ml-specific technical debt, how to measure it, some ideas on how to handle it, this paper has spawned a lot of great tools - http://papers.nips.cc/paper/5656-hidden-technical-debt-in-ma...
Adversarial instance and outlier detection - https://github.com/SeldonIO/alibi-detect
And of course a whole suite of tools and research beingn done on data, pipeline, and model management, optimizing deployment and serving, explainability and interpretability, all in the service of applying enterprise-ready, engineering-grade rigor to data science.
And then you read something like this and want to tear your hair out.