News for hackers. Stuff that matters.
Re:Test machine learning the right way: regression testing by saladtoes on Tuesday July 06, 2021 @08:53AM (#27747836)
“The latter being when the training or test data follows a different distribution to the in-operation data”

This form of ML bug is the most challenging to catch. The true in-operation distribution is often unknown which makes testing for such bugs a very challenging problem. Any thoughts on this?

Re:Test machine learning the right way: regression testing by mklond on Tuesday July 06, 2021 @09:00AM (#27747919)
Thanks for your comment. The whole field of run-time monitoring is concerned with this problem. It's a tough one to crack when the distribution changes are subtle, but you can and should at least check simple data attributes for consistency.
Re:Test machine learning the right way: regression testing by mklond on Tuesday July 06, 2021 @08:50AM (#27747795)
Would love to hear what other testing techniques people use for machine learning? Are there great testing frameworks out there that people use?