News for hackers. Stuff that matters.
Re:AI Models Predict Breast Cancer with Radiologist-Level Ac... by twanvl on Tuesday June 25, 2019 @01:57PM (#20276942)
> The problem is that they fail differently than humans do

That is a great argument for giving such a model as an aid to a human doctor. Together they will be better then either one alone.

Re:AI Models Predict Breast Cancer with Radiologist-Level Ac... by onlyrealcuzzo on Tuesday June 25, 2019 @02:13PM (#20277114)
In Thinking, Fast and Slow -- the author details a double blind trial where the did this. It was worse with humans and AI than with just AI. Humans think they can use AI as a guide and move it in the right direction. But the movements they made, on average, were bad.
Re:AI Models Predict Breast Cancer with Radiologist-Level Ac... by notahacker on Tuesday June 25, 2019 @03:01PM (#20277658)
Surely in this type of instance (looking at a scan to answer a yes/no question) the human and AI act independently, with the computer being a useful aid because it separately picks up a few of the human's false negatives. Assuming false negatives are a lot worse than false positives, this can only be a good thing.
Re:AI Models Predict Breast Cancer with Radiologist-Level Ac... by chongli on Tuesday June 25, 2019 @10:29PM (#20281365)
If they lead to an unnecessary mastectomy then false positives are pretty bad. Not as bad as dying, obviously, but still a severe blow to a woman's identity and sense of self worth.

It's going to be a hard pill to swallow if you have to tell a woman "sorry, we removed your healthy breast because the computer made a mistake."

Re:AI Models Predict Breast Cancer with Radiologist-Level Ac... by glangdale on Wednesday June 26, 2019 @12:26AM (#20281908)
I think the idea of "screening" is that you don't just race off to a mastectomy the minute some AI model goes off. Of course, putting more false positives through a fallible process of review does run the risk you speak of.
Re:AI Models Predict Breast Cancer with Radiologist-Level Ac... by AstralStorm on Wednesday June 26, 2019 @03:49AM (#20282725)
It does cause unnecessary biopsies for sure. And some stress on the patients.
Re:AI Models Predict Breast Cancer with Radiologist-Level Ac... by irq11 on Wednesday June 26, 2019 @12:11AM (#20281837)
Even a false positive that leads to telling the patient that they may have cancer is bad. It leads to a life-long anxiety for many people.
Re:AI Models Predict Breast Cancer with Radiologist-Level Ac... by stubish on Wednesday June 26, 2019 @05:41AM (#20283341)
It sounds like a smart hospital would run a patient through both human and AI screenings separately, and a different doctor to examine both results and evaluate the discrepancies. This way you would keep the strengths of both approaches, lowering the failure rates, and depending on the countries health care funding can be good business from the hospital's POV as they get to charge for the extra work as well as the better success rates to drive business.

And I wonder what happens if you apply machine learning to looking at the difference between AI and human screening results.

Re:AI Models Predict Breast Cancer with Radiologist-Level Ac... by bitL on Tuesday June 25, 2019 @02:19PM (#20277190)
Radiologists are really bad at detection, even after many years of study. That's quite often due to coarse level of details of scans when only large tumors can be observed or recognized with some certainty. Surpassing humans there is not so difficult, but improving accuracy from e.g. 32% to 34% doesn't really sound like a win :(
Re:AI Models Predict Breast Cancer with Radiologist-Level Ac... by carlmr on Tuesday June 25, 2019 @03:03PM (#20277689)
2% more accuracy could still be millions of people if it's a common enough cancer like breast cancer.
Re:AI Models Predict Breast Cancer with Radiologist-Level Ac... by schwurb on Tuesday June 25, 2019 @07:53PM (#20280520)
> 32% to 34% doesn't really sound like a win :( We are talking about human lives here, not about beating some CPU benchmark. Detection improvement by 2% is huge in almost any sickness.
Re:AI Models Predict Breast Cancer with Radiologist-Level Ac... by baq on Tuesday June 25, 2019 @03:19PM (#20277842)
remember when ensembles were the cool word before they got erased from collective consciousness and replaced with deep things? it can't even be a decade, was it 2012 or something?
Re:AI Models Predict Breast Cancer with Radiologist-Level Ac... by krisoft on Tuesday June 25, 2019 @05:18PM (#20279061)
They haven't got erased, but more like subsumed? If you use dropout to train your model that is basically equivalent with using an ensemble of deep neural networks.
Re:AI Models Predict Breast Cancer with Radiologist-Level Ac... by irq11 on Wednesday June 26, 2019 @12:13AM (#20281844)
That is not even close to the same thing.

If you train an ensemble of models with random dropout, you have an ensemble. Models trained with dropout will still have significant variation from run to run.

Re:AI Models Predict Breast Cancer with Radiologist-Level Ac... by sooheon on Wednesday June 26, 2019 @12:24AM (#20281903)
> That is not even close to the same thing.

It's a common interpretation: https://arxiv.org/abs/1706.06859

Re:AI Models Predict Breast Cancer with Radiologist-Level Ac... by irq11 on Wednesday June 26, 2019 @06:53PM (#20290099)
There may be a paper on it, but it’s not a common view.

In particular, this paper neglected to do the obvious thing: ensemble networks trained with dropout. It improves performance over dropout alone.

Re:AI Models Predict Breast Cancer with Radiologist-Level Ac... by rerx on Tuesday June 25, 2019 @06:00PM (#20279472)
Why shouldn't you employ an ensemble of deep neural networks?
Re:AI Models Predict Breast Cancer with Radiologist-Level Ac... by salty_biscuits on Wednesday June 26, 2019 @01:39AM (#20282177)
Correlated errors. Naive averaging will lead to overconfidence and it is not trivial to model the correlation. Boosting is worth a shot though.
Re:AI Models Predict Breast Cancer with Radiologist-Level Ac... by rerx on Thursday June 27, 2019 @12:50PM (#20296161)
My point was that ensembles of deep neural networks are commonly used and yield higher accuracies.
Re:AI Models Predict Breast Cancer with Radiologist-Level Ac... by AstralStorm on Wednesday June 26, 2019 @03:51AM (#20282738)
More importantly, what happens if you put a radiologist opinion (or multiple) in such an ensemble?
Re:AI Models Predict Breast Cancer with Radiologist-Level Ac... by acangiano on Tuesday June 25, 2019 @02:24PM (#20277241)
This. The argument was never to replace doctors. These are valuable tools that augment what doctors can do.
Re:AI Models Predict Breast Cancer with Radiologist-Level Ac... by TeMPOraL on Tuesday June 25, 2019 @04:09PM (#20278345)
No, the point very much is to eventually replace doctors. You just can't easily get there before first going through a doctor-machine cooperation period.

Automation is a friend of society, but is not a friend of individuals working particular jobs. I think doctors are acutely aware of that.