We don't use deep learning - we use Biophysical models. We hate using the term "AI". This is a very challenging discipline to explain to VCs.
Also, speaking to point 2 here - the "value" of building tools for ultrasound is often dismissed by VCs because "ultrasound isn't used for screening or diagnosis". This is an insane perspective from our position when we are practically based within hospitals, collaborating strongly with radiologists and medical oncologists who work with ultrasound on a daily basis.
We are very embedded within the hospital and look to understand the clinicians workflow and decision making processes first, as well as understanding what's possible given the hurdles involved in data access (which can still be tricky even when you are through IRBs and ethics).
We have found that telling VCs the reality about working with hospitals and doctors can often limit their excitement about your company prospects. Our success to date has largely been as a result of doctors and hospitals who believe in us, see the value in what we are doing. They have put time and effort into collaborating because they are impressed with what we have been able to do results wise by bootstrapping as a small team, rather than as a VC funded shiny startup.
In a weird way i would say that at it's best times medtech can be one of the "purest" industries to work in. By this i mean ultimately your technology works or it doesnt (at least from the medical communities perspective - again VCs are a different story). There are obviously exceptions to this (Theranos anyone) and there are issues around the 510K process but on the whole there is a big price to pay for making unsubstantiated claims (say compared to aspirational lifestyle marketing).
The paper specifically talks about mammography, it does not claim to replace a complete diagnosis.
> 2) Speaking frankly as a radiologist myself, tests like mammograms aren't even that good in terms of overall diagnosis. Thats why ultrasound, tomosynthesis and MRI are often used as supporting evidence and/or alternative exams.
From the abstract: "2) successfully extends to digital breast tomosynthesis"
> 3) There is controversy over the overall utility of mammograms, particularly in the screening context.
> It strikes me that the people that push these "radiology is ripe for disruption" [...]
The paper, which I just skimmed over, does not read hyperbolic, for that we'll have to wait for popsci journalists.
OTOH, if one leaves the 1st world context, any type of successful diagnosis automation in medicine is a blessing for areas where you simply don't have enough trained medical staff.
I also interact with AI/ML researchers all the time. Most of them are typically some combination of: 1. Poorly informed about the appropriate context and utility of medical imaging. 2. Trying as hard as they can to push AI/ML as the most important technology in medicine today. 3. Pursuing a very task-specific project which they claim is massively generalizable in some (incorrect) way.
My engineering mind just can't come up to terms with this. Why wouldn't you collect all information you possibly can? You can always ignore irrelevant data you have, but you cannot consider data that you don't have!
The closest I've been to rationalizing this: diagnosing is a stochastic process so complex (and with the search space so large) that the random noise in extra data is likely to point you towards wrong directions. Plus you can always collect more data afterwards if your initial diagnosis turns out to be wrong. This is of course very simplified, but it makes sense.
However, I just can't turn off my inner voice from screaming "more data is always better". I guess that's why I'm not an MD :)
Everything we know about human psychology says you can't.
From a decision theoretical viewpoint, you would certainly want all the information you could get. For humans running a business in today's medicolegal environment, it's a very different set of issues:
1) Collecting information costs time and money.
2) Making good decisions requires the most precious resource of all, which is doctor brain-time. There isn't enough of it to spend on information with little probability of benefit.
3) If you get sued for malpractice, the unneeded data you collected probably would not have helped the patient, but it could help the attorneys arguing that you missed something. Juries struggle to understand the cost of false positives.
Even though there are valid issues here, doctors don't always make the right tradeoff in this regard. Oftentimes, I think it is more an issue of lack of training or experience that leads a doctor to consider a test to be unneeded. In the case of mammography, if doctors spend too much time doing screening and not enough time doing diagnosis, their screening performance degrades, which I think is due to a lack of feedback on their decision making[1].
You run a screening company. You take people at high risk of lung cancer -- people who smoke a lot and have smoked a lot for many years -- and you provide low dose CT scans of their lungs.
Bob comes in. You scan his lungs and you find spots.
What do you do now?
You're probably going to start providing treatment to Bob. Will this help Bob live longer? Will it improve his quality of life? It might not.
https://blogs.bmj.com/bmjebmspotlight/2019/02/15/understandi...
Hopefully get other tests done, to confirm diagnosis.
I'm with GP here. I can't understand this attitude either. Having more information should never make you more wrong. This holds for uncertain information, because uncertainty can be quantified and tracked (if you're not doing this, then you're doing voodoo, not science).
I can see two reasons why you wouldn't want to gather more information in medical context. One, many tests carry risk to patient's health and well-being, so there's no point of doing them if that risk outweighs the expected value of evidence gathered. Two, I suspect that gathering information also gathers legal obligations and risks to doctors.
Those other tests involve things like "needle biopsy" -- they shove a needle through your chest into your lung into the suspect tissue to get a sample. This carries risk. We can justify that risk if it saves life. But this is the problem with screening -- often it doesn't save life (of course, it depends on the type of screening).
https://www.radiologyinfo.org/en/info.cfm?pg=nlungbiop
> Having more information should never make you more wrong
But you can see how having lots of low-quality information could make someone more wrong -- these are not clear signals, because if they were it wouldn't be a problem. These are almost noise. We're taking data from a large population ("4 in 100 people with this result have this disease") and trying to apply it to the individual, and when we try to get more information we subject this person to more radiation in scans or invasive procedures or both. We increasing the risk, but not necessarily saving life.
> there's no point of doing them if that risk outweighs the expected value of evidence gathered
Yes, this is exactly the balance that doctors are making. They're looking at all cause mortality and seeing if life is saved.
In the short term, more information might cause harm because doctors are risk averse & scared of lawsuits and err on overbiopsy/overtreat, and many of our treatments aren't as good as we think they are, and all of this makes patient anxious.
In the long term, turning the information firehose on full blast means we can work out which incidental findings are best ignored or pursued and overall more data will help us.
The problem is that it is unethical to do #2 in the short term even if it is the long term ethical thing to do.
I completely get your attitude, I think I agree with you overall and if I was not this lazy I could comb through my bookmarks and find the studies supporting what you said.
But I was just responding to your comment in the context of the paper linked. Which, at least when skimming over it, does not read like what you (IMHO, rightfully) criticize in the broader debate.
And yes, read the first paragraph as a tongue-in-cheek response, we both know that overgeneralizations don't help any debate ;)
I wonder how long people think about these sorts of claims before posting them.
Do they really think an AI is more likely to appropriately interpret an MRI scan (and all the anatomic, physics and pathophysiological data therein contained) in the context of a specific clinical work-up more easily than triage patients the way a family practitioner or ER doctor does?
I wasn't just commenting on the abstract presented. I was commenting on the comments I see here, as well as comments I see related to similar papers all the time.
I also interact with AI/ML researchers all the time. Most of them are typically some combination of: 1. Poorly informed about the appropriate context and utility of medical imaging. 2. Trying as hard as they can to push AI/ML as the most important technology in medicine today. 3. Pursuing a very task-specific project which they claim is massively generalizable in some (incorrect) way.
In fact, the area of medicine most in need of 'disruption' imho is healthcare enterprise software. Doctors are literally killing themselves because the interfaces they have to deal with on a daily basis are so appallingly poor.
Of course, the solution is less technological than political as it wouldn't take much to come up with better software than the legacy alternatives but you'd have a very hard time getting past the entrenched relationship interests of crony bureaucrats that run hospital administrations.
1) Mammograms are not interpreted in a vacuum. In fact mammograms are usually the first in a long line of tests before a breast cancer or other diagnosis is ultimately made. In fact, it's probably more accurate to refer to mammography as a screening exam for which patients need a biopsy rather than a diagnostic test for cancer (there are rare exceptions, but overall this point holds).
2) Speaking frankly as a radiologist myself, tests like mammograms aren't even that good in terms of overall diagnosis. Thats why ultrasound, tomosynthesis and MRI are often used as supporting evidence and/or alternative exams.
3) There is controversy over the overall utility of mammograms, particularly in the screening context. Radiologists more than anyone would like the sensitivity and specificity of these studies to be higher.
It strikes me that the people that push these "radiology is ripe for disruption" or "AI outperforms radiologists" hyperbolic arguments are clearly people that have never seen the inside of a clinic. I'm sure they love this rhetoric though when pitching to VCs or sitting around the conference table coming up with 'breakthrough ideas' to turn into power-points for the other administrators.