The term the article uses is more accurate. A hallucination can be entirely disjoint from the context it appears in. Confabulation is a confident belief in an erroneous recall that one will attempt to defend and rationalize. That's exactly what's happening with an LLM. They choose the wrong token or series of tokens, essentially failing to remember correctly, and then present it confidently and make rationalizations about why it's correct.

> make rationalizations about why it's correct.

Its not rationalising, its statistically choosing the next token. Also, generally the LLM doesn't get defensive (although its not always the case.)

Its not like a dementia patient not finding their keys and gradually convincing themselves that someone stole them instead. Because the alternative, that they are loosing cognitive faculties, is too horrific to acknowledge.

This is why its so annoyingly problematic. too much humanising, and not enough witnessing someone with dementia "confabulating"

It is noise, and nothing else.

Maybe, but tell me this, how do you rationalize or reason without using words or symbols of any kind?

by using logical fallacies.

This comment isn't saying anything. It says "LLMs statistically choose the next token" and, because of that, it can't be anything else.

We know how the LLMs were trained, so re-stating it doesn't help in anything. The point is that after the LLMs are trained they behave in certain ways and it can be helpful to say something about how it behaves.

For example, we can talk about how a linear regression can or cannot capture the causal effect of X1 over Y. "It's not capturing the causal effect. It's just minimizing the squared error" is an unhelpful statement.