I really think the whole "hallucination" thing is a terrible framing. All that gpt-4 does is spew out bullshit. Now, it applies a bunch of statistics to attempt to spew out believable bullshit and those statistics cause it to sometimes accidentally be correct. The base assumption with this should be that it's wrong, and so we should instead be framing it as like sometimes accidentally producing "actualities" or something.