Are you concerned about privacy or bad advice? If the latter, you don't have to blindly just do whatever the LLM suggests.

I can't imagine an individual who both has their health records (pertaining to some issue) and also doesn't have the ability to understand the context.

If I feed in 15 crown/root canal invoices into AI asking it it's opinion, it will tell me I have a very bad mouth. This does nothing to inform me, and will lead to the expected AI response of: "Here are some practices in your area specializing in oral surgery".

There's a reason it's advised by doctors to not share health information. practices/doctors proscribe a regimen/treatment plan, and have to assume in good faith the patient will follow it. Imagine if every doctor heard "Well, AI told me..."

> If I feed in 15 crown/root canal invoices into AI asking it it's opinion, it will tell me I have a very bad mouth

This is just blatantly made up. This doesn't even slightly resemble something an LLM would say

I can, because I've been that person. I got a blood panel done recently. Didn't hear anything back from my doctor because everything was in the OK range. But I still wanted to actually understand the results. And yes, I could have spent time researching what each individual market meant. Or I could ask an LLM, which gives me the same information in seconds. Now obviously if I planned to take an action like starting a supplement or something based on that information I would verify it, but the LLM is still providing value there.