We have a much bigger problem with overdiagnosis than underdiagnosis, and the latter is not likely to be solved with this sort of scattershot data collection. The former is very likely to be inflamed.

The difference is that you actually hear about the Diagnosis X That Was Missed, but you never hear about That Simple Blood Test That Should Never Have Been Drawn but that Showed Positive for a Marker Common In Disease But Also Found in the General Population so They Needed a Biopsy to Rule Out, and Spent Hundreds or Thousands if Dollars and Lost Several Days of Work For Nothing At All. The latter is much, much, more common.

The “miss” is usually due to missing something in the modern deluge of data coupled to inadequate appointment time/resources, not lack of data availability.

For a real life example: heard about a patient die the other day. Reviewing his records with a student afterwards I saw his potassium had been climbing in his last two panels (though it hadn’t yet gotten high enough to be a red flag), and he didn’t get the blood gas that should have been done as per protocol. Thing is, the data was there, the rules were there, but someone with enough time to meaningfully review it was not. Would more piles of detritus data to sift through made that more likely, or less?

I also remain an ML skeptic for the time being. ML analysis of patient data is a big sophisticated version of retrospective analysis, which for purely statistical reasons is highly, highly unreliable.

I just finished (the book) Everybody Lies. It seems to me using the doppelganger approach (as mentioned in the book) would at least be helpful.

A couple months ago I spent a good number of hours over a couple week at local hospital waiting for a close friend's mum to recover from a stroke. I could be mistaken but the only "data analysis" I noticed was taking place in the heads of the doctors and nurses.

The same goes for the physical therapy facility. That is, no one - not even the insurance company - was using similar cases as a guide. Yes, at a high general level they were. But using real data didn't seem to be on their radar.

This void made family decision making stressful and difficult.

That is correct: the analysis of large data sets is opaque to patients, and application of the evidence base to specific patients happens in the heads of physicians.

Patients are not part of the decision making process, and therefore not privy to the analysis, unless there is a decision to be made that requires their input (eg, two reasonable roads forward that depend on their preferences, or on their risk-tolerance.)

The process is made opaque to patients intentionally.

Yes. But at no point did I sense the options being offered were based on doppelganger type of data. Or even (the doctor saying) "here's the data...here's our interpretation...here are the recommendations and pros and cons..."

Nothing like that. No continuity between docs or floors/wards. Just everyone kinda making it up as they go.

It just struck me as another reason why outcomes aren't optimized, and costs, relatively, inflating.

I'm not suggesting doctors are replaceable. But the doppelganger approach to (big) data seems to be ideal.

> "here's the data...here's our interpretation...here are the recommendations and pros and cons..."

I understand. That’s part of what is intentionally opaque. We don’t speak like that to anyone but other medical professionals. There’s a lot of Dunning-Kruger in what people think of their ability to parse hc information. As someone who grew up with serious chronic disease, I’ve been on both sides of this fence. It’s not something you really “get” until you’ve had to care for patients.

> costs inflating

Several Health Econ studies have shown that about 70% of hc costs are driven by new technologies (implants, procedures, patented meds.) It has nothing to do with how docs communicate with one another.

And how we communicate is also opaque to patients. Not intentionally, it’s just not a conversation for patients, any more than any two technicians in a software company talking/arguing technical details are having a convo meant for the ears of customers.

I think it is good to remain skeptical about medical ML, but I don't think it should be because of the algorithms or that retrospective analysis is flawed. In most retrospective medical research, the data sizes are relatively small. Likewise current ML needs huge amounts of labeled data to be accurate, and patient data just isn't available to ML experts in the volumes necessary. The question I have is would dramatically increasing the size of available patient data improve ML analysis? Is it the case that some of the flaws of retrospective analysis are actually a result of too little data especially in the context of ML? I don't know the answer, but I do know we need to fix the data availability problem if we want to find out. How do we fix the data availability problem? Not sure either, it a complex problem.

Not sure how this proves me wrong. What it does prove is that analysis of data should not be left (entirely) to humans, who are inherently flaky. Dark irony of this situation is that _more_ data from additional test had a chance of saving the patient.

The way it should have worked: based on previous labeled examples, computer system should have noticed the uptrend in potassium and flag this client for blood gas, and then make classification based on available and _additional_ data.

It’s not the blood gas that would have saved him; just seeing the potassium trending up would have done it. Thing is, there was a context where that mattered. In others, it wouldn’t have. That’s why you need someone capable of interpretation to -look- at it. EMRs have done more to harm patients than help them, now that everyone constantly copy-pastes old patient hx before adding their notes (so nothing can be said to have been missed), people are constantly shifting through piles of be and missing the pertinent information.

To rephrase: algos intelligent enough to be useful in the manner you imagine are a long way off from existing, but data deluge that harms patient care is -already here-. Don’t worsen the deluge until you improve the ability to digest and present it.

That EMR “decision aids” aren’t anywhere near sophisticated enough to tell the difference in the myriad of situations we have is why docs fight so hard against those automated “flags.” We get piles of them, all the time. QI people call it alarm fatigue, they know it’s a thing, and yet... they do it anyway, because it sounds (like in your post) like the obvious thing to flag.

It’s not that AI is inherently incapable of getting there. It’s that when it does is a long way off. Medicine looks a lot simpler from the outside.

I agree with you and are suspicious of the power of increasing raw data in the near term. I think there are a few dumb things we can do that have a chance at being useful. The usefulness of something like this would have to be extremely narrowly focused. For example, it can look at only potassium trends. We would first need to just monitor people for long periods of time and see what happens. We might see certain combinations of simple markers that make additional tests worth the risk and minimize false alarms. After this research, would we turn on the alarms and start worrying people.

Thanks for the insight. I wonder though about the difference between scattershot data collected in a hospital setting, vs personalized data collected over the normal, healthy course of a person's life. Establishing relevant baselines would go a long way towards eliminating noise and flagging real outliers.

I have nothing but agreement with all your points.