Troubling Trends in Machine Learning Scholarship

Re:Troubling Trends in Machine Learning Scholarship by drewm1980 on Wednesday July 11, 2018 @03:15AM (#17505012)

"The ubiquity of this issue is evidenced by the paper introducing the Adam optimizer [35]. In the course of introducing an optimizer with strong empirical performance, it also offers a theorem regarding convergence in the convex case, which is perhaps unnecessary in an applied paper focusing on non-convex optimization. The proof was later shown to be incorrect in [63]."

I strongly disagree here. Researchers, please keep trying to prove that your proposed optimizers are correct, even if you can only manage to prove it for a subclass of problems, and even if it risks publishing an accidentally incorrect proof.

[ Parent | on HN ]

Re:Troubling Trends in Machine Learning Scholarship by cosmic_ape on Wednesday July 11, 2018 @04:08AM (#17505241)

Adam may be not the clearest example. I'd say the clearest example is the original GAN paper, where "theorems" where completely unrelated to what they were doing and trivial at the same time.

>>and even if it risks publishing an accidentally incorrect proof.

No. If a researcher has even a slightest doubt about the correctness of their proof, they should either spend the time to verify it or talk to an expert. There is no lack of experts in convex optimization in particular. Or, instead of publishing the proof, one could just say "we believe this is provable by these methods". But one can not try and shift the burden of verification to the readers, while claiming the results. This is not how trust is built.

[ Parent | on HN ]

Re:Troubling Trends in Machine Learning Scholarship by blt on Wednesday July 11, 2018 @04:06AM (#17505232)

Yeah I read that proof as more of a reassurance than an emphasized part of the paper. If the method does not even work on convex problems, it's highly suspicious as a tool for nonconvex problems.

[ Parent | on HN ]

Re:Troubling Trends in Machine Learning Scholarship by AstralStorm on Wednesday July 11, 2018 @03:53AM (#17505171)

Yes, but why do it on paper when you can use an automated prover instead, since you're proving properties of an algorithm?

It smells of laziness...

[ Parent | on HN ]

Re:Troubling Trends in Machine Learning Scholarship by shoo on Wednesday July 11, 2018 @07:07AM (#17505975)

can you cite any non-trivial examples where people writing papers include machine-checked proofs , in fields of study unrelated to the study of machine-checked proofs?

[ Parent | on HN ]