Deep learning outperformed dermatologists in melanoma image classification task

Re:Deep learning outperformed dermatologists in melanoma ima... by plus on Tuesday April 30, 2019 @01:55PM (#19790272)

Since this is a journal focused on cancer and not machine learning, I can understand why the editors would see this paper as being worthy for for publication. Unfortunately, many of the readers will read the paper uncritically.

If possible, you should write a critical response to this paper, focusing on its methodological flaws, and send it to the editors. It doesn't have to be long; critical response are usually a couple pages at most. This is likely the most effective way of removing (or at the very least, heavily qualifying) bad science from research journals.

[ Parent | on HN ]

Re:Deep learning outperformed dermatologists in melanoma ima... by 0xab on Tuesday April 30, 2019 @02:18PM (#19790510)

This is a huge problem throughout science, not just ML. As scientists, we're rewarded for publishing cool new things that work, not for pointing out things that don't or for pointing out flaws in existing papers. If the point is to get people to not read one bad paper, it's just a waste of my time. Most papers are false and a lot of them should never have passed review.

If the authors actually wanted to do good ML research, they could always have reached out to a decent ML researcher who could have told them all of this. There's no shortage of us. The journal could have reached out to an ML reviewer. Why wouldn't they? But no one did, because the results look good and so they send it off to press and it's good for both the authors and the journal to have something that is hype-worthy. It's just the sad reality of modern science.

[ Parent | on HN ]

Re:Deep learning outperformed dermatologists in melanoma ima... by PierredeFermat on Tuesday April 30, 2019 @02:50PM (#19790764)

It's amazing that a similar concern is raised/discussed here just couple hours ago: https://news.ycombinator.com/item?id=19788088

Any chance we could connect over email or something?

[ Parent | on HN ]

Re:Deep learning outperformed dermatologists in melanoma ima... by mehrdadn on Tuesday April 30, 2019 @02:43PM (#19790709)

> Most papers are false and a lot of them should never have passed review.

Do you mean this literally or is this a metaphor to illustrate the point? If you actually mean most papers are false it'd be nice to see a link on that!

[ Parent | on HN ]

Re:Deep learning outperformed dermatologists in melanoma ima... by michaelhoffman on Tuesday April 30, 2019 @02:46PM (#19790731)

John Ioannidis claims that "most published research is false" based on some rather dubious assumptions.

https://www.annualreviews.org/doi/abs/10.1146/annurev-statis...

[ Parent | on HN ]

Re:Deep learning outperformed dermatologists in melanoma ima... by DataWorker on Tuesday April 30, 2019 @04:47PM (#19791783)

I agree with him although the accuracy of that statement is partially based on how “published research” is defined. Operational definitions and measurement are themselves much of the problem.

[ Parent | on HN ]

Re:Deep learning outperformed dermatologists in melanoma ima... by michaelhoffman on Tuesday April 30, 2019 @02:02PM (#19790351)

How to Publish a Scientific Comment in 1 2 3 Easy Steps

http://frog.gatech.edu/Pubs/How-to-Publish-a-Scientific-Comm...

I agree that a formal comment is best although not necessarily easy. A comment on PubPeer is easier but it will probably only be seen by those with the PubPeer extension.

I do machine learning in computational biology and cancer. The issues described in the parent comment are known among experts. It’s too bad so many others don’t know or care.

[ Parent | on HN ]

Re:Deep learning outperformed dermatologists in melanoma ima... by chris_va on Tuesday April 30, 2019 @03:12PM (#19790941)

Thank you for that link, it was a joy to read

[ Parent | on HN ]

Re:Deep learning outperformed dermatologists in melanoma ima... by Florin_Andrei on Tuesday April 30, 2019 @02:22PM (#19790551)

I mean, if it's an interdisciplinary study, you may want to get advisers from all sides to look at it before you publish, no?

[ Parent | on HN ]