But bias has a negative impact too.

It’s not unusual for clinicians to interpret differently based on patient history and other risk factors.

The same CT scan is going to be interpreted very differently for a patient with no history of cancer versus one who went into remission 1 year ago.

That's not bias, it's use of relevant context. If you're going to have an LLM doing diagnoses, you're definitely going to feed it that sort of context.

It's a matter of degree though.

Do you just tell the AI "this person had cancer?", or "this person went into remission 1 year ago?" or "this person was treated with 4 lines of chemo, partial remission for 1 line, non-response for 2nd line, strong response for cycles 1-3 for 3rd line, but then discontinuation due to tolerability, then 4th line complete remission after 6 cycles plus two courses of radiotherapy"?

Theoretically you could do this, but capturing all this and correctly interpreting it is far harder for AI than for a human who has been directing treatment the entire time.

Meanwhile for late childhood to to mid adulthood, 99% of my visits have been a doctor listening to me for 2 minutes tops while applying what amounts to a flow chart of how to diagnose and treat something like strep. Yes a doctor is best, but remember in many places we're competing with "no access do doctor so just pray it doesn't go to Rheumatic fever."

Very often asking the _right_ questions is hardest part especially in a non-typical cases. Statistical ML models tend to do well on high-probability regions of data that are densely sampled in training and are less good dealing with outlier cases. I am curious to see how GPT-4 deals with hard atypical cases.

Did you mean to write bias can have a positive impact, too?

Any model or human will suffer from some degree of uncertainty. You can never say anything with 100% certainty.

If you know a patient had cancer before, the chance that they have cancer again is much higher usually than the general population. This is very valuable for diagnosis.

It's good to have a prior/bias to check more closely here and err on the side of caution.

It's good to check for the more likely explanations first when you have multiple competing hypotheses and limited capacity for testing.

> if training data has bias

I think most, if not all, training data has bias. Removing bias from training data is a challenge in and of itself that I don't think we've solved yet. I worry that there isn't enough incentive there for us to solve it and some minorities will be left behind.