Why are deep learning technologists so overconfident?

Re:Why are deep learning technologists so overconfident? by legulere on Wednesday August 31, 2022 @02:55PM (#32666732)

I think the big issue is that we have bad intuition about what works well with machine learning and what doesn't, leading to both under- and overconfidence at the same time. When I was at university I came across machine learning in medical imaging. The job of radiologists seemed obvious to be a classic machine learning problem that should be easily solved by the upcoming possibilities. In the end other stuff materialized.

[ Parent | on HN ]

Re:Why are deep learning technologists so overconfident? by colinmhayes on Wednesday August 31, 2022 @03:09PM (#32666922)

The radiology stuff is happening. My father is an administrator at a hospital. They went from having 3 radiologists on staff to 2 because they use ai to screen which increases productivity. These things don't have to be all or nothing, the hospital is saving hundreds of thousands a year now because radiologists are so expensive.

[ Parent | on HN ]

Re:Why are deep learning technologists so overconfident? by trention on Wednesday August 31, 2022 @04:29PM (#32667928)

The "radiology stuff" is not happening. Data from BLS (https://www.bls.gov/oes/current/oes_nat.htm)

Employed Radiologic Technologists in the US:

2014 - 193,400

2021 - 216,380

The growth is larger than the overall population growth in the same period. And that is in a very outsourceable occupation.

[ Parent | on HN ]

Re:Why are deep learning technologists so overconfident? by Wojtkie on Wednesday August 31, 2022 @05:36PM (#32668676)

To be fair, Radiologic Technologists are different than a Radiologist. The former is a certification requiring a 2 year degree, the latter requires a Doctorate of Medicine which takes an additional 4 years at least on top of 4 years of undergrad. Then you need 4 years of residency and an additional sub-speciality year on top of that to be a Radiologist.

So, very different.

[ Parent | on HN ]

Re:Why are deep learning technologists so overconfident? by trention on Thursday September 01, 2022 @02:24AM (#32672695)

The latter is a subset of the former in BLS statistics and I am using the larger category only because BLS apparently only started counting Radiologists as a separate subcategory in 2019.

[ Parent | on HN ]

Re:Why are deep learning technologists so overconfident? by robertlagrant on Wednesday August 31, 2022 @06:25PM (#32669139)

Yes. Techs and doctors are different!

[ Parent | on HN ]

Re:Why are deep learning technologists so overconfident? by cma on Thursday September 01, 2022 @02:06PM (#32680534)

Two points (your conclusion that AI hasn't had much effect yet may still be right):

Population growth isn't everything. The uninsured rate dropped from 12% to around 8% in that time period. Not many uninsured people get radiologist services unless it is an emergency.

And cheaper AI-assisted Radiologists could increase the overall employment of Radiologic Technologists.

[ Parent | on HN ]

Re:Why are deep learning technologists so overconfident? by lucidrains on Wednesday August 31, 2022 @03:27PM (#32667126)

out of curiosity, which hospital is this?

[ Parent | on HN ]

Re:Why are deep learning technologists so overconfident? by gpm on Wednesday August 31, 2022 @03:00PM (#32666801)

Do you happen to know what stopped the radiologist stuff from happening?

I know I read press releases ages ago saying "hey look, we can do this machine learning", but I haven't really heard anything about it since.

[ Parent | on HN ]

Re:Why are deep learning technologists so overconfident? by labcomputer on Wednesday August 31, 2022 @04:04PM (#32667628)

Part of it is medical regulation.

For example, only a human doctor can make a "diagnosis" or write a prescription. Even just making "treatment recommendations" (e.g., a more-effective treatment or one with less side effects than the current one), which would then be reviewed and approved by an appropriately-licensed physician, is a problem.

Not having worked in the radiology space, my naive hunch is that it has to be framed as a "screening tool" which "helps the physicians find cancers." (Which can be justified to regulators because ML tends to have higher recall than humans on these tasks, albeit lower precision).

In theory, ML helps doctors spot cancers they wouldn't ordinarily see. In practice, it probably means that the physicians still technically "review" all images, but only give a cursory glance to those not flagged by the system.

[ Parent | on HN ]

Re:Why are deep learning technologists so overconfident? by naijaboiler on Wednesday August 31, 2022 @05:31PM (#32668634)

I am a doctor and an AI practioner. A radiologists job is far far more than pathology detector. No AI will replace the image recongition aspect of their job in my lifetime. And that's the most automatable aspect of their job. The human and legal aspects of their job is a whole different beast. At best, technologies will allow radiologists to better or more efficient

[ Parent | on HN ]

Re:Why are deep learning technologists so overconfident? by sdenton4 on Wednesday August 31, 2022 @03:10PM (#32666930)

My impression was that medical data access for training and eval was the really big thing holding it back... You wind up with deep learning folks obsessing over datasets with tens of examples.

[ Parent | on HN ]