I have a friend and his wife had some weird stomach problems that the local doctors couldn't figure out. It sounds like the doctors basically ignored that a problem existed because they couldn't think of anything.

Eventually they had to go to the nearest big city (city they're in now is 60K people; city they went to has 900K people). The doctors there figured it out pretty quickly. And they also did not have good things to say about the local doctors. (paraphrase) "I'm not surprised they didn't find anything. They look for the simple solution they know and if that doesn't work they give up."

Additionally, I went to a doctor for a problem in the same big city and he read (and printed out) a webmd article for me.

Anyway. So, today people are stuck with human doctors who have no idea how to fix anything but the simple obvious problems. Personally, I wouldn't mind replacing those people with an AI as I'm not sure it's going to make any difference. There's always going to be specialists and if the AI doctor fails you then you can go to the specialist. But just like today, you're going to have to be your own medical researcher / advocate if the service you get doesn't actually fix your problem.

Wait until you have a rare disease before you say something like you want AI help diagnose you (or make decisions about your healthcare for that matter—the US is going to be a mess if they go this route). 7% of the general population collectively has a rare disease, so it’s not uncommon to have a rare disease. Most of them are classified as very rare.

I personally have 2 rare diseases affecting my peripheral nervous system, and one of them is very rare. The very rare disease is believed to have also caused my type 1 diabetes.

The wild part? The very rare disease, which I had prior to my type 1 diabetes diagnosis, which was also undiagnosed at the time, was blamed as “type 1 diabetes related complications” specifically autonomic neuropathy by human doctors.

Do you think AI would get this right? The answer is an obvious no.

Also, I am in my early 30s. My endocrinologist was surprised that I was 30 and was making a big deal about it (as in “you survived and you shouldn’t be alive” sense). It was never statistically likely that I would be alive, which is why AI should never be used.

So do you think AI should be making health decisions in a healthcare system that already has a ton of human error, and is trained on those errors?

I mean, the third leading cause of death in the US is believed to be preventable medical errors: https://www.bmj.com/content/353/bmj.i2139

I don't understand your argument.

You're saying that humans are unacceptably bad. Then why is it a problem to replace them with also unacceptably bad AI. Like, either way you're not getting the treatment you need.

At least if the AI fails you, then nobody is going to get their ego stepped on if you go to get a second opinion. Whereas with a doctor you have to go through a few uncomfortable conversations and then hope that the next guy isn't going to ignore you because they don't want to sour their relationship with your first doctor.

> Do you think AI would get this right? The answer is an obvious no.

Today? Sure. But really one of the most plausible ways in which AI could be more effective than human clinicians is in rare diseases. It assumes a massive input of data of course, but clinicians are biased against this in ways that are hard to do anything effective against. This is why so many people with rare conditions spend years before getting a proper diagnosis.

Getting that data is an extremely hard problem. In general the data is low quality, incomplete, poorly coded, and full of errors. Cleansing the data to the point where it could be used to train a useful decision support algorithm would require a tremendous amount of human labor by expert clinicians. This isn't something that can be automated.

Okay ... how do doctors (individually or as a whole) get better at doctoring? Either we're saying they look over the data OR we're saying that they somehow just mystically know better as time goes on.

Who cares if it turns out that we can't automate the formation of the AI. But we're doing the same work to continually educate and improve doctors already. At least with the AI it won't die of old age once it starts to get really good at what it's doing.

And maybe it turns out that the best that AI can do is duplicate a middle level doctor who isn't really interested in self improvement. If that's the case then at least everyone can get cheap and fast health care that's the same frustrating quality that we're stuck with now. Only with the AI you can say that it's dumb and everyone will believe you. With a doctor you have to fight past "impugning the reputation of the profession" in order to get a real diagnosis.

Doctors are getting better at doctoring mostly through simple things like checklists and evidence-based medicine treatment guidelines. Those reduce variance and the risk of error. Improved interoperability between clinical systems also has potential to reduce errors by making sure all members of a patient's care team have complete access to the patient's entire record.

Diagnosis isn't particularly difficult or time consuming in most routine cases after gathering the necessary data. If you see a jagged line on the x-ray then the patient has a fractured bone. If the A1c level is 7.5% then the patient has diabetes. Etc. Improving diagnostic accuracy will help in rare cases and is worth doing, but that won't have much impact on the health care system as a whole.

I don't think I understand your argument.

You at first appear to say that gathering data and automating things is hard and might not be possible.

Then here you're saying that checklists, evidence-based guidelines, and improved interoperability between systems are all the source of doctor improvements.

These are all arguably forms of AI that have been created through data gathering. Yes, a checklist is a very boring AI, but it's taking some knowledge out of the heads of a group of people and then making it available for someone else to use. The missing step seems to be that instead of having a person look at a piece of paper and execute the checklist and evidenced based treatment, having a computer do it.

You're missing the point. There's no cost effective way to provide the computer with the complete, high quality data necessary to execute the checklists.

I hope I can be forgiven for missing the point when this is the first time (as far as I can tell ... perhaps you said something in a different thread) you've mentioned cost. At first it was "this cannot be automated". Then it was "well it happened, but it's okay because it's written down on paper and not put in a computer".

Now we're talking about cost. Cost where (at least in the USA) going to the ER for a broken bone can cost thousands of dollars. Cost where (at least in the USA) a birth (something people from all walks of life go through) has a minimum of the patient spending $10K and regularly gets into the $50K and up range (not even talking about ultrasounds etc leading into the birth).

I cannot believe that we're really talking about data collection being too expensive for the medical industry. Maybe nobody wants to be the person to pick up the bill. But it's going to be a rounding error of a rounding error compared to all the other money that goes through that industry.

The data gathering and entry can't be automated. It could be done manually, but at a very high cost (not a rounding error). Commercial insurers and Medicare certainly wouldn't be willing to pay. As a practical matter it's just not worth doing in the vast majority of cases.

Thanks again for explaining it so well. Usually me and you do not agree on healthcare issues, but this is one area where we seem to be on the same page.

As for diagnosing rare or very rare diseases, yes, it is possible, as you said. However, if you search for the rare disease I have in PUBMED, there are only 111 journal articles that have been written about it, and some are in foreign languages. The information on it is very sparse. I also doubt that it would be diagnosed by AI, as it effectively looks like "diabetes complications" (I have type 1 diabetes) and even though I had autonomic neuropathy at age 5, the onset of symptoms was very insidious.

> [...] to the point where it could be used to train [...] require a tremendous amount of human labor by expert clinician.

This is how it has been done for decades at this point (at least in some systems), and yes it is expensive. To really tackle things like above rare disease interactions, it would require at least an order of magnitude change, this is at least plausible; but would require some systemic changes.

Correct and thank you. Usually me and you don't agree on healthcare matters, but on this, at least we do. Your response is on point, and is exactly the issue.

I have a lot of concerns about AI moving into spaces that involve human life and death.

With that said, I think you've highlighted one possible outcome, but it makes a few assumptions that might play out differently depending on how the tech is implemented.

Imagine, for sake of argument, that the AI is just "part of the team". It doesn't replace a doctor, it doesn't have authority over doctors, it's just another signal. If applied in this way, then you can still:

- Ask for a second opinion

- Go to a doctor/team that doesn't rely on AI

- Consult a different / (better?) AI

I realize you're focusing on an outcome where AI replaces humans in the field, but I'd hope humanity can find an implementation that can both leverage the benefits of the tech while not throwing out the core disciplines of medicine in the process. It's reasonable to be pessimistic about this given the tech-run-amok issues prevalent in 2021, but I still have some hope.

Also imagine the possibilities of AIs sharing knowledge with each other. This has its own semi-terrifying implications, but I don't necessarily think we should conclude that the only outcome is a single AI to rule all AIs (and doctors).

Of course, I could be wrong as well. But I don't think all roads lead to doom.

You could say all these same things about any time in human history where some process got automated, standardized, mechanized, or computerized. Indeed, many people did. They're called Luddites.

People always get scared that automation will completely remove the human touch from a process. They forget that people will still design, understand, troubleshoot, maintain, improve, be inspired by, learn from, and innovatively transform their new automated devices and processes.

The future has always ended up being people and machines working together, and I think it will continue to be that way for a long time. There will not be an AGI that replaces human thinking in our lifetime.

Instead of a pile of machines, imagine a vast opaque bureaucracy, where any individual participant doesn't have any decision-making power and must follow a prescribed set of steps. I think this is partially what fuels nightmares of your average modern luddite.

If you follow a happy path in such a system, everything is fine. But if you don't, it gets Kafkaesque. Check out a first hand account [0] for flavor.

[0] https://slatestarcodex.com/2017/08/29/my-irb-nightmare/

Heartless, rule-based, unappealable decisions have been a feature of human society for as long as there has been organized bureaucratic society. It is not the case that soft-hearted human bureaucrats are all that stands in the way of the Kafkaesque nightmare.

If we want better social and political decisions, we have to engage in society and politics. The AI will follow what we decide, as it always does.

I don't think AI will make us dumber, it will allow us to work faster and more abstractly. Doctors have no incentive to regurgitate an AI prediction; they're the ones liable for care. Doctors will continue to second guess AI while they use its predictions as one of many data points for their decision.

This is actually kind of what I'm hoping for. The saying is "when you hear hoofbeats think horses not zebras." But that's boring and it turns the doctor into a bored and checked out professional who just constantly sees the same symptoms day in and out and prescribes the same treatment. When something interesting does show up, they're too used to the status quo to actually address the issue.

Let the AI prescribe all of the horse treatments. The doctor can then spend their time imagining more fanciful diagnosis that we're centuries away from teaching the AI how to deal with. Then when the primary treatment fails, instead of trying to convince an overworked and checked out doctor that you've got some other problem, they'll actually already have a list of possible alternative issues that they were really hoping they would get to investigate this time around.

James Bridle wrote a rather good book titled "New Dark Age" that covers the continual increase in reliance on algorithms and often non-human-verified data in our decision making processes in all aspects of our lives. This impacts scenarios from the individual level when people drive off roads and into hazards following their GPS blindly, all the way to global markets behaving in unexpected ways due to trade algorithms being unable to account for never before seen changes to conditions that don't fit within their existing parameters.

There's some similar work that frames some of this around cognitive artifacts. There are processes people perform that create an artifact in their mind they can use for other purposes of problem solving.

The model leaves a sort of precense on their consciousness, an artifact,, and allows them to reason in ways aided by those artifacts. When you stop creating and using those artifacts and depend on technology to act in place of them, you lose some creative aspects you would normally be able to connect or utilize in other mental models.

For some things I think it's good for technology to act as black boxes we don't have to understand, just know that we can understand it if we need to. Other things are good to understand even if you can automate it. How you choose and differentiate which of those is an incredible challenge in my mind. What do I commit to memory and integrate into my mental model of the world and what do I just reference and use when I need it.

I think that a lot, perhaps most, AI diagnoses would involve a feedback loop that tracks the patient's health outcomes.

Here are some examples of feedback: - The AI misses a cancer that gets worse and is later detected on a blood test. - The AI detects a cancer on an x-ray but a (more accurate) CT or biopsy indicates that there is no cancer

These kinds of feedback loops should allow the clinical performance of AI to be measured and hopefully improved over time.

This has been going on for 25ish years now, but the iteration is quite slow and there are systemic barriers to speeding it up.

>? Imagine being misdiagnosed by a computer and the clueless human health care workers don't or won't question it because the computer said so. You'll be one the phone talking to an AI which is looking at records generated by AI full of data compiled by AI. Why pay costly humans when you can replace them with a computer.

Has already been happening.

I highly doubt this. Source: work in the industry.

I see, I reacted on a other comment and I can't delete this one.