Re:A Worrying Analysis of Recent Neural Recommendation Appro...
by YeGoblynQueenne on Monday July 22, 2019 @07:10PM
(#20503134)
Well. In theory, one should be able to reproduce (implement) an algorithm described in a paper just by reading the paper. It's better if the algorithm is typeset as, you know, an algorithm, but if the algorithm can be better described in plain English...
Besides, _ideally_, machine learning reseach is not about building a system but about making a theoretical claim that is supported by empirical results. The main claim should be about a property of the algorithm: its time complexity, its convergence, its correctness, etc. Experiments should directly test the theoretical claim and verify it in practice.
... ideally. In practice most machine learning papers are vehicles for a little table that compares your system with a baseline that you chose based on the fact that your system beats it (or you wouldn't have put it in the paper). Whether your experiments prove anything beyond the fact that you beat some benchmark is not really important.
And so whether someone publishes a reproducible paper is of no consequence because the only point of papers is to pass review and convince someone that the researchers should stay in the (research) money. In any case, things move so fast that most of the time nobody wants to reproduce your old and unsexy paper from last week.
Then some busybody comes along and points out the fact that you're wearing no clothes. Like in the paper above.
Besides, _ideally_, machine learning reseach is not about building a system but about making a theoretical claim that is supported by empirical results. The main claim should be about a property of the algorithm: its time complexity, its convergence, its correctness, etc. Experiments should directly test the theoretical claim and verify it in practice.
... ideally. In practice most machine learning papers are vehicles for a little table that compares your system with a baseline that you chose based on the fact that your system beats it (or you wouldn't have put it in the paper). Whether your experiments prove anything beyond the fact that you beat some benchmark is not really important.
And so whether someone publishes a reproducible paper is of no consequence because the only point of papers is to pass review and convince someone that the researchers should stay in the (research) money. In any case, things move so fast that most of the time nobody wants to reproduce your old and unsexy paper from last week.
Then some busybody comes along and points out the fact that you're wearing no clothes. Like in the paper above.