News for hackers. Stuff that matters.
Re:Machine Learning Reproducibility Checklist [pdf] by DrNuke on Friday December 27, 2019 @08:54AM (#21891301)
This is aimed at production or critical applications, though, not forefront or blue-sky research. In the former case, we need a shared & agreed framework to make sure everyone from everywhere gets statistically comparable results, with this checklist helping us in that sense. In the latter case, it is open field and we are looking for agreeable results approximation before a method, which will be devised later to fit concordant results.
Re:Machine Learning Reproducibility Checklist [pdf] by MAXPOOL on Friday December 27, 2019 @09:58AM (#21891649)
I agree with you in principle. I still think reproducible should be a goal even in pure research in blue sky machine learning for the following reasons:

Even basic research is can be sensitive to omitted parameters, setups and starting conditions. Honest mistakes, accidental omissions and failing to spot sensitivity to parameters happens all the time. Writing "A clear explanation of any assumptions" is rarely comprehensive. Its easy to miss things and become blind to some fine details.

Discovering and documenting of a new interesting phenomenon and dynamics is also part of basic research. Publishing experimental discoveries without explanation should be always reproducible.