I dislike the term "hallucinations" because I feel it also anthropomorphizes the process too much. Unfortunately, "random garbage output" is too many words, but that's closer to what I meant everywhere I used that word.

> Larger ones (e.g. GPT-4) can even spot their own hallucinations.

I've not yet been convinced that this is actually what is happening from the examples I've seen. It all looks to me like more "random garbage output" that "feels correct" but isn't provably correct. Most examples I've seen so far look too much like "Stochastic Crow Mode" [1]. It is prompts and questions that are doing much more work on the humans reading them (and our interests in anthropomorphizing them or mythologizing them) than the LLMs answering them.

[1] https://fediscience.org/@ct_bergstrom/110182336553459017