The above is a common problem that UX research is interested in. I'm not sure how much it is solved, but it goes well beyond medical fields.
There was a paper a while ago about an effort at something like this. They got the AI going and noticed that the first thing it did was classify all the x-rays by race of the patient. Then they freaked out, gave up on their original project, and wrote their paper about how AI is inherently evil.
The human will still have to look at the x-ray to see if the AI missed something. 95% accuracy is not good enough, those 5% of cases are what most of their training is for, missing it can mean a lost human life. Maybe it can be used to speed up obvious diagnoses, but it cannot be used to filter and rule anything out. The amount of time a radiologist will spend looking at the x-ray will probably not be reduced, so I don't think there's money to be saved here.
A useful productivity tool could be to examine datasets after the radiologist found nothing, as a way to double-check their reading. This won't reduce costs but might marginally improve patient outcomes. Radiologists in first-world medical systems don't really miss a lot of stuff, though.
And of course for simple obvious non-life-affecting stuff like broken bones and dental x-rays, you don't need radiologists now either. Your son's x-ray was probably not looked at by a radiologist.
Isn't this very context dependent? E.g., a delay in lung cancer diagnosis may be a very big deal, but much less so for something like prostate cancer
As the CDC says on its page for prostate cancer screening[0]:
>Screening finds prostate cancer in some men who would never have had symptoms from their cancer in their lifetime. Treatment of men who would not have had symptoms or died from prostate cancer can cause them to have complications from treatment, but not benefit from treatment. This is called overdiagnosis.
>Prostate cancer is diagnosed with a prostate biopsy. A biopsy is when a small piece of tissue is removed from the prostate and looked at under a microscope to see if there are cancer cells. Older men are more likely to have a complication after a prostate biopsy.
[0] https://www.cdc.gov/cancer/prostate/basic_info/benefits-harm...
From "Discrepancy and Error in Radiology: Concepts, Causes and Consequences" (2011) (https://www.ums.ac.uk/umj081/081(1)003.pdf):
> In the 1970s, it was found that 71% of lung cancers detected on screening radiographs were visible in retrospect on previous films [4,6].
> The “average” observer has been found to miss 30% of visible lesions on barium enemas [4].
> A 1999 study found that 19% of lung cancers presenting as a nodular lesion on chest x-rays were missed [7].
> Another study identified major disagreement between 2 observers in interpreting x-rays of patients in an emergency department in 5-9% of cases, with an estimated incidence of errors per observer of 3-6% [8].
> A 1997 study using experienced radiologists reporting a collection of normal and abnormal x-rays found an overall 23% error rate when no clinical information was supplied, falling to 20% when clinical details were available [9].
> A recent report suggests a significant major discrepancy rate (13%) between specialist neuroradiology second opinion and primary general radiology opinion [10].
> A recent review found a “real-time” error rate among radiologists in their day-to-day practices averages 3-5%
> In patients subsequently diagnosed with lung or breast cancer with previous “normal” relevant radiologic studies, retrospective review of the chest radiographs (in the case of lung cancer) or mammogram (in breast cancer cases) identified the lung cancer in as many as 90% and the breast cancer in as many as 75% of cases [11].
> A Mayo Clinic study of autopsies published in 2000, which compared clinical diagnoses with post-mortem diagnoses, found that in 26% of cases, a major diagnosis was missed clinically [11].
For the case of AI analysing x-ray photos, the obvious solution would be a system that can tag photos with information about what AI thinks is going on there. And this information could be passed to the human.
This could save a ton of time and help reduce cases where radiologists missed obvious things.
My son once broke his arm. I brought him to ER, they made the photo but the two people who looked at the photo said there is nothing wrong with the arm. I asked for the copy of the photo.
A week later the swelling did not subside so I took the photo to another doctor and he pointed out an obvious fracture line.
There are many ways to deploy automation and I wonder why everybody tries to shoot for removing humans altogether when most of the time this is literally asking for problems.