That's a real shame. It's been years since "The Unreasonable Effectiveness of Data" has been published and out in the world. You'd think that using a large N for statistical learning methodologies would be standard practice by now.

Do Hospitals keep radiological images? Are they researching this stuff? Presumably they'd have a large enough sample size.

Yes, hospitals keep imaging on their PACS systems and VNAs (Vendor Neutral Archives). Large systems have millions of images, though not necessarily same body part. Very large systems have millions of the same body part alone.

Getting images is one thing. Getting labels is harder. Getting annotations on regions-of-interest is yet harder. Often the labels are stuck in unstructured data (notes.)

When I was doing my startup (https://www.dochuddle.com/) we trained our classifier and object detector on 1.2 million images. We worked with a large sovereign on getting the images, reports, and annotations.

Just dealing with 20+ TB of images was a job in its-self...but it was indeed unreasonably effective. The success was only technical and ML success.

Even harder -- commercializing it effectively after accounting for legal/contract costs. Yet harder -- getting past conflicts of interest inherent in the US medical system. We were not successful on this front. Possibly too early (we started in 2014)

Interesting; thanks for sharing. It also seems like AI/ML could be used to extract labels from unstructured notes.

Yes, you can use NLP. One trick would be to use a medical vocab like https://www.snomed.org/ if anyone is doing this

https://en.wikipedia.org/wiki/SNOMED_CT#Semantic_tag

Actually, a real shame is a total lack of really big (millions of records) data sets on pretty much anything medical. And this is of course due to our love with privacy. Probably millions of people died too soon because of that, and many more will follow in the future.

I'd agree that millions suffer due to a lack of care, or a lack of affordable care, or a lack of early diagnosis.

I'd disagree this is due to privacy. IMHO it is due to conflicts of interest in major medical systems (certainly in the US) where incentives are skewed away from efficiency towards more billing.