I see this conversation pretty frequently and I think the root of it lies in the fact that we have mental heuristics for determining whether we need to fact check another human because they are a bullshitter, an idiot, a charlatan etc, but most people haven’t really developed this sense for AIs.

I think the current state of AI trustworthiness (“very impressive and often accurate but occasionally extremely wrong”) triggers similar mental pathways to interacting with a true sociopath or pathological liar for the first time in real life, which can be intensely disorienting and cause one to question their trust in everyone else, as they try to comprehend this type of person.

> The smart people I've seen using ChatGPT always double check the facts it gives.

I don't like being told lies in the first place and having to unlearn it.

It doesn't help that I might as well have just gone straight to the "verification" instead.

I don't know. The other day I was asking about a biology topic and it straight up gave me a self-contradicting chemical reaction process description. It kept doing that after I pointed out the contradiction. Eventually I got out of this hallucination loop by resetting the conversation and asking again.

It's smart but can also be very dumb.