News for hackers. Stuff that matters.
Re:Deep learning outperformed dermatologists in melanoma ima... by 0xab on Tuesday April 30, 2019 @02:18PM (#19790510)
This is a huge problem throughout science, not just ML. As scientists, we're rewarded for publishing cool new things that work, not for pointing out things that don't or for pointing out flaws in existing papers. If the point is to get people to not read one bad paper, it's just a waste of my time. Most papers are false and a lot of them should never have passed review.

If the authors actually wanted to do good ML research, they could always have reached out to a decent ML researcher who could have told them all of this. There's no shortage of us. The journal could have reached out to an ML reviewer. Why wouldn't they? But no one did, because the results look good and so they send it off to press and it's good for both the authors and the journal to have something that is hype-worthy. It's just the sad reality of modern science.

Re:Deep learning outperformed dermatologists in melanoma ima... by PierredeFermat on Tuesday April 30, 2019 @02:50PM (#19790764)
It's amazing that a similar concern is raised/discussed here just couple hours ago: https://news.ycombinator.com/item?id=19788088

Any chance we could connect over email or something?

Re:Deep learning outperformed dermatologists in melanoma ima... by mehrdadn on Tuesday April 30, 2019 @02:43PM (#19790709)
> Most papers are false and a lot of them should never have passed review.

Do you mean this literally or is this a metaphor to illustrate the point? If you actually mean most papers are false it'd be nice to see a link on that!

Re:Deep learning outperformed dermatologists in melanoma ima... by michaelhoffman on Tuesday April 30, 2019 @02:46PM (#19790731)
John Ioannidis claims that "most published research is false" based on some rather dubious assumptions.

https://www.annualreviews.org/doi/abs/10.1146/annurev-statis...

Re:Deep learning outperformed dermatologists in melanoma ima... by DataWorker on Tuesday April 30, 2019 @04:47PM (#19791783)
I agree with him although the accuracy of that statement is partially based on how “published research” is defined. Operational definitions and measurement are themselves much of the problem.