So “hallucination” hews close enough to have good explanatory powers.
> In psychology, confabulation is a memory error defined as the production of fabricated, distorted, or misinterpreted memories about oneself or the world. […] Confabulation occurs when individuals mistakenly recall false information, without intending to deceive.
This seem a deep statement that keeps getting repeated, but it doesn't mean anything. The probabilistic model that is used to decide the next token could be arbitrarily complex, including encoding knowledge (or just asking a panel of experts).
It seems pretty self evident that the model in fact encodes knowledge, just in a very lossy way and recall is also flawed.
I mean, isn't this what humans do all the time? Bullshitting random topics on the Internet, except humans tend to add disclaimers like "I am not a lawyer but" and stuff.
No? Most humans don’t randomly vomit text based on what sounds good.
> Bullshitting random topics on the Internet, except humans tend to add disclaimers like "I am not a lawyer but" and stuff.
Which shows a much higher level of understanding, both of the field (which may be flawed), and of their own understanding of the field (which they point out).
An LLM does not to that, it doesn’t just repeat potentially wrong hearsay or incorrect memories (let alone having actual understanding and knowledge of the field), it confidently writes out delusions.
Unless humans were given a task? e.g. taking exams while un-prepared.
My kid usually gives me a long description of imaginary stuff based on the name only or brief intro. It's very fun when finally the real deal was revealed.
Couldn’t exactly be otherwise given how young GPT is. ChatGPT was released a bit under 7 months ago.
> Is it a legit concept or just "the dog ate my homework" type of excuse for anything?
It’s an analogy for how LLMs work. An LLM does not know anything, it just adds tokens probabilistically based on the previous tokens.
So essentially it always hallucinates (makes shit up as it goes along, if you prefer).
Thanks to the model it’s generally quite credible, and often even lines up with actual reality, but it should not be confused for knowledge.
That’s why it will confidently give you citations it just made up, to papers or decisions it’ll happily make up as well (though less and less credibly as things get closer to hard facts).