I actually can't wait to start reporting to GPT, how much crazier can it be?
Just as a made up example, say you asked "what is the capital city of england", and the model had seen similar questions in it's training data that were answered with "the capital city of X is Y", so it starts word-by-word generating this type of response "the capital city of england is", but it then turns out the model doesn't actually know the answer (i.e. this partial response context doesn't cause it to predict the correct answer), so it blithley predicts the next word as whatever it's inner machinations come up with, maybe "buckingham palace" or "flavor town".
"Hallucination" seems a poor way to describe it, nor is it lying since there's no bad intent ... it's basically "starting to speak before brain engaged", a bit like a game show contestant being a bit too enthusiastic and hitting the "answer" button without actually having a fully thought out answer in mind.