Re:A.I. Systems Echo Biases They’re Fed, Putting Scientists ...
by ska on Wednesday November 13, 2019 @01:57PM
(#21527958)
> And the AI's always fail this test.
This isn't quite true as it has been successful in limited scopes since the 90's (i.e. well before the existence and rise of deep learning). See for example microcalc TP rates on CAD breast screening that was beating average radiologists back then, but not on general screening. This is somewhat specialized though, and there are important constraints to screening also, as opposed to diagnostic radiology.
But you are right in an important way - it isn't a feature for the current throw-a-ton-of-data-at-a-deep-model approach that a lot of "AI companies" are built around, and I agree they are mostly knocking on the wrong doors in medical imaging.
Importantly, if you are trying to replace people at a relatively simple task and make it cheaper your approach will look much different than the case you describe, where you are trying to learn unusual configurations and long-tail sort of things.
ML in medical imaging is complicated by a lot of factors, but people that think a little transfer learning and a few hundred case reports is going to get somewhere useful are being incredibly naive. Without exception, all the promising work I have seen has involved painful and expensive labeling and a lot of modeling and pre-processing effort.
This isn't quite true as it has been successful in limited scopes since the 90's (i.e. well before the existence and rise of deep learning). See for example microcalc TP rates on CAD breast screening that was beating average radiologists back then, but not on general screening. This is somewhat specialized though, and there are important constraints to screening also, as opposed to diagnostic radiology.
But you are right in an important way - it isn't a feature for the current throw-a-ton-of-data-at-a-deep-model approach that a lot of "AI companies" are built around, and I agree they are mostly knocking on the wrong doors in medical imaging.
Importantly, if you are trying to replace people at a relatively simple task and make it cheaper your approach will look much different than the case you describe, where you are trying to learn unusual configurations and long-tail sort of things.
ML in medical imaging is complicated by a lot of factors, but people that think a little transfer learning and a few hundred case reports is going to get somewhere useful are being incredibly naive. Without exception, all the promising work I have seen has involved painful and expensive labeling and a lot of modeling and pre-processing effort.