This is why I believe these bots should never be taken too seriously, at least until there is a lot more transparency around their sources of information. Until there are bots which are trained on highly verifiable sources of information that's open sourced.

I don't think it's wise to just "trust a doctor" but can you imagine the influence of such a system. Imagine if the wrong person becomes the CEO of OpenAI and people are using it for medical advice, all of a sudden it starts recommending people take opiate based medication for many, many ailments? We have this with doctors today, who would be exposed to a lot more accountability than a mega-corp with access to massive legal teams / robots.

I see this as actually being a real existential risk to large swathes of the populace.

I hate to say it, but I'd already be taking every single thing returned with a grain of salt, and probably doing that forever.