It’s not like that at all. Hallucinations are complete fabrications because the weights happened to land there. It has nothing to do with how much thought or double checking there is.

You can trick an LLM into “double checking” an already valid answer and get it to return nonsense hallucinations instead.