Re:Hallucination Is Inevitable: An Innate Limitation of Larg...
by rapatel0 on Friday February 09, 2024 @08:48AM
(#39314801)
This is pretty obvious no? LLMs are basically a lossy compression of their dataset. Being lossy, they will necessarily have error (hallucinations). Furthermore, the underlying data is human approximation of truth. Therefore it will have error as a well. Shall we publish a paper on the linked list?
Re:Hallucination Is Inevitable: An Innate Limitation of Larg...
by rhelz on Friday February 09, 2024 @09:36AM
(#39315219)
Hmmm....couldn't you argue it the other way as well? I mean, a hallucination isn't just an incomplete fact present in the input set--it is a claim which wasn't in the input set.
That's not what we'd expect from a lossy compressor; we would expect incomplete or missing answers, but we wouldn't expect it to give us claims which were not in the input set.
That's not what we'd expect from a lossy compressor; we would expect incomplete or missing answers, but we wouldn't expect it to give us claims which were not in the input set.