News for hackers. Stuff that matters.
Re:ChatGPT: Fear Litany by masklinn on Tuesday June 20, 2023 @03:17AM (#36400720)
> This "hallucination" come along a lot recently.

Couldn’t exactly be otherwise given how young GPT is. ChatGPT was released a bit under 7 months ago.

> Is it a legit concept or just "the dog ate my homework" type of excuse for anything?

It’s an analogy for how LLMs work. An LLM does not know anything, it just adds tokens probabilistically based on the previous tokens.

So essentially it always hallucinates (makes shit up as it goes along, if you prefer).

Thanks to the model it’s generally quite credible, and often even lines up with actual reality, but it should not be confused for knowledge.

That’s why it will confidently give you citations it just made up, to papers or decisions it’ll happily make up as well (though less and less credibly as things get closer to hard facts).

Re:ChatGPT: Fear Litany by Sharlin on Tuesday June 20, 2023 @03:26AM (#36400780)
Which is why "hallucination" is really the wrong word to use, "confabulation" would be more proper. But "hallucination" has stuck because it's the word used back when people first figured out the trick of running image classifiers "in reverse" to generate images from noise.
Re:ChatGPT: Fear Litany by masklinn on Tuesday June 20, 2023 @04:06AM (#36401022)
Sure but nobody knows the word “confabulation”, and lying / making things up implies intent.

So “hallucination” hews close enough to have good explanatory powers.

Re:ChatGPT: Fear Litany by Sharlin on Tuesday June 20, 2023 @04:49AM (#36401334)
Confabulation is unintentional, FWIW:

> In psychology, confabulation is a memory error defined as the production of fabricated, distorted, or misinterpreted memories about oneself or the world. […] Confabulation occurs when individuals mistakenly recall false information, without intending to deceive.

Re:ChatGPT: Fear Litany by masklinn on Tuesday June 20, 2023 @05:21AM (#36401554)
Yes, which is why I agree that it’s a better term. That’s not the issue.
Re:ChatGPT: Fear Litany by Sharlin on Tuesday June 20, 2023 @05:42AM (#36401704)
Ah, I misinterpreted your previous comment!
Re:ChatGPT: Fear Litany by gpderetta on Tuesday June 20, 2023 @04:20AM (#36401108)
> It’s an analogy for how LLMs work. An LLM does not know anything, it just adds tokens probabilistically based on the previous tokens

This seem a deep statement that keeps getting repeated, but it doesn't mean anything. The probabilistic model that is used to decide the next token could be arbitrarily complex, including encoding knowledge (or just asking a panel of experts).

It seems pretty self evident that the model in fact encodes knowledge, just in a very lossy way and recall is also flawed.

Re:ChatGPT: Fear Litany by seba_dos1 on Tuesday June 20, 2023 @04:30AM (#36401195)
It sure does encode some knowledge, because it's a language model and languages already do so on their own. It's far from what you'd usually call a "knowledge model" though.
Re:ChatGPT: Fear Litany by est on Tuesday June 20, 2023 @04:32AM (#36401205)
> it just adds tokens probabilistically based on the previous tokens

I mean, isn't this what humans do all the time? Bullshitting random topics on the Internet, except humans tend to add disclaimers like "I am not a lawyer but" and stuff.

Re:ChatGPT: Fear Litany by masklinn on Tuesday June 20, 2023 @05:24AM (#36401576)
> I mean, isn't this what humans do all the time?

No? Most humans don’t randomly vomit text based on what sounds good.

> Bullshitting random topics on the Internet, except humans tend to add disclaimers like "I am not a lawyer but" and stuff.

Which shows a much higher level of understanding, both of the field (which may be flawed), and of their own understanding of the field (which they point out).

An LLM does not to that, it doesn’t just repeat potentially wrong hearsay or incorrect memories (let alone having actual understanding and knowledge of the field), it confidently writes out delusions.

Re:ChatGPT: Fear Litany by est on Tuesday June 20, 2023 @05:54AM (#36401787)
> Most humans don’t randomly vomit text based on what sounds good.

Unless humans were given a task? e.g. taking exams while un-prepared.

My kid usually gives me a long description of imaginary stuff based on the name only or brief intro. It's very fun when finally the real deal was revealed.

Re:ChatGPT: Fear Litany by seba_dos1 on Wednesday June 21, 2023 @05:47AM (#36416458)
That's absolutely right. That said, people don't usually take exam output of unprepared students and expect it to be useful :)
Re:ChatGPT: Fear Litany by number6 on Tuesday June 20, 2023 @05:30AM (#36401618)
"As a Language Model" is the new "I am not a lawyer"