I've been working in data science or in academia for most of the last decade and would argue that the primary benefits of using statistical models for automation are that they are fast and cheap compared to a trained professional, which frees those trained professionals up to spend more time on the difficult judgment calls. I've seen very few examples of superhuman performance on complex, practical tasks (although it is very satisfying to train a model to that level).

> And the AI's always fail this test.

This isn't quite true as it has been successful in limited scopes since the 90's (i.e. well before the existence and rise of deep learning). See for example microcalc TP rates on CAD breast screening that was beating average radiologists back then, but not on general screening. This is somewhat specialized though, and there are important constraints to screening also, as opposed to diagnostic radiology.

But you are right in an important way - it isn't a feature for the current throw-a-ton-of-data-at-a-deep-model approach that a lot of "AI companies" are built around, and I agree they are mostly knocking on the wrong doors in medical imaging.

Importantly, if you are trying to replace people at a relatively simple task and make it cheaper your approach will look much different than the case you describe, where you are trying to learn unusual configurations and long-tail sort of things.

ML in medical imaging is complicated by a lot of factors, but people that think a little transfer learning and a few hundred case reports is going to get somewhere useful are being incredibly naive. Without exception, all the promising work I have seen has involved painful and expensive labeling and a lot of modeling and pre-processing effort.

> I've come to believe that "bias" is the primary reason these AI's perform so poorly. They are basically "biased towards the known stuff" in layman's terms. At least in my field, we need AI's that are able to make connections that humans cannot.

AI, at least current AI, cannot do this. I'm really confused at the way you say AI perform poorly because they're biased towards "known stuff". We train AI on known data. AI being biased towards "known stuff" is fundamental to the way our current AI technology functions.

You really sound like this topic of bias in AI is new to you. Here’s a real world example: a medical school used to be male-only. Recently they began admitting women. More recently, they began running “AI” trained on past student applications to figure out which applications were likely to be successful students (where successful == graduating normally). Not surprisingly at all, it turned out that being female wasn’t associated with being a good student there, because it was an uncommon attribute of previous students.

This is a poor result. Would you argue that the model should not be adjusted?

The field of AI is not new to me. Quite the opposite, I find that many people in this thread are making fundamental misunderstandings about what AI and machine learning are about. Hence the commenters like the one above which say AI is limited because it is biased towards past data, which sounds ridiculous to anyone that works with AI because using past data to train the model is fundamental to how AI works.

Yes, if people feed in a bad data set the results are going to be bad. Like how a pentagon project tried to use AI to classify Russian vs NATO vehicles. The photos of the former were taken on a sunny day and the latter on an overcast day. The result was that their model was just detecting the average brightness of the input.

But what happens when the data is valid, but people don't like the results because it doesn't match their worldview? Altering the model to match one's world view is not elimination of bias, it's deliberately introducing bias.

In fairness here, we're not talking about data that doesn't match some world view or some such nonsense. This is diagnostic imaging. The data just doesn't have any world views. It's a tumor, or not. It's malignant, or not. The stenosis calculation is what it is. None of these things are liberal or conservatives. They don't have a narrative.

So when we're evaluating these systems, (again, systems that always fail due to bias), our only concern from a diagnostic viewpoint are patient-centric metrics. The most important being improving outcomes.

By way of illustration, let's suppose a woman goes for a mammogram. Now a radiologist and these AI's can all identify tumors in the resultant DX study at about the same point in said tumor's development. This leads to our current 2 and 5 year survivability rates. Consider thoughtfully here, the name of the game is not liberalism, or conservatism, the object in our game is to improve those survivability metrics.

Here's the thing though, due to the fact that the AI is biased towards things we already know, using it doesn't allow us to identify problems any earlier. Being able to spot issues earlier would improve 2 and 5 year survivability metrics, but current state of the art AI's won't do that. This is the problem in a nutshell.

The problem is not that the resultant data does not fit the "narratives" of our radiologists.

I'm not certain you understand that this is not a political argument. The AI's that vendors are attempting to apply towards problems in this domain simply don't improve healthcare outcomes. If they did, we would aid in deploying them. Without hesitation.

Now my assertion is that these AI's fail to improve healthcare outcomes due to the bias inherent in the data they use to train on. Again, that's not a political statement. It's just fact. The AI's seem mostly trained to internalize, and replicate as output, all of the well known patterns in the data. This is why these AI's are of severely limited use.

I don't think you're understanding the point I'm making. I agree, we should make our models that are as accurate as possible. But this view is not universal. There are some that point out that prioritizing accuracy may create disparities in the results of the model. A model that is meant to predict recidivism will flag African American more often than Whites and men more often than women because the former of these groups re-offend at higher than the latter.

Some see this fact as problematic, and advocate that we should deliberate bias the results in order to create a more equal outcome. And the arguments of whether or not we should bias our models are frequently political in nature. The complaints that AI is biased is frequently because it produces results that some people find are problematic, but the underlying pattern it's identifying is true. It's really that the AI matches patterns without our cultural sensitivities about what topics we're supposed to tiptoe around.

Well again, you're making political arguments.

The AI results are the way they are, because the data that is input tells the AI to spit out those results. In other words, the AI isn't telling you anything that a human didn't tell it to tell you. The valuable insights are insights that no human saw coming.

So let's take your industry as an example with which you can maybe better relate. Humans give the AI data that says that africans and men have higher recidivism. The AI then spits out results that says that africans and men have higher recidivism. Basically spitting back the information that we already gave it. OK, so far, so expected.

Now here's the thing, the valuable insight would be for the AI to suss out some counter intuitive result that no one would ever have gotten. The result that this particular african, or this particular man will have lower recidivism. That's the valuable insight. That insight saves the enterprise money that the enterprise would not have saved otherwise.

My point is that current AI's can't do that, which makes them not terribly useful. At the moment, they just parrot what any human would tell you anyway. So why pay 2 mil for that? Can you see what I'm saying? This problem is even more acute in diagnostic imaging, because the AI is not improving outcomes. So what, exactly, would we be paying 2 mil for? It's absolutely problematic for me to go out and put an enterprise on the hook to the tune of 2 mil for a black box that sits in a room, looks cool, and tells us things we already know. (In fact, the box would tell us things we told it to tell us.) We can save the money, by a 30 dollar digital recorder, and have it play the doctors' diagnoses back to the doctor after every patient meeting if that enterprise is really that much into the parroting.

My problem with the bias in the AI's I've seen is not political, it's practical. This stuff, as it is, is not terribly practical because it is too biased to offer any truly unique actionable insights that add value which was not there previously.

At this point we are diverging almost entirely from questions about whether AI is biased, and more about whether AI is cost effective.

The allegations of bias in AI are frequently about inter-grouo differences in things like job advertisement, credit ratings, parole decisions, etc. which are not explicitly political but the allegations of often political in nature.

Sure, there's not much controversial about predicting tumor growth. But are the allegations of bias in AI about models predicting tumor growth? No, they're usually about the topics I listed above.