News for hackers. Stuff that matters.

A comment on Hallucination is inevitable: An innate limitation of large language models. Parent comment

Re:Hallucination is inevitable: An innate limitation of larg... by AlexandrB on Sunday February 25, 2024 @01:29PM (#39503359)
> They clearly go far beyond this and generate completely novel ideas that are not in the training data.

There's a case where this is trivially false. Language. LLMs are bound by language that was invented by humans. They are unable to "conceive" of anything that cannot be described by human language as it exists, whereas humans create new words for new ideas all the time.

[ Parent | on HN ]
Re:Hallucination is inevitable: An innate limitation of larg... by pixl97 on Sunday February 25, 2024 @05:26PM (#39505549)
Uh, I believe you're really confused on things like ChatGPT versus LLMs in general. You don't have to feed human language to an LLM for them to learn things. You can feed wifi data waveforms for example and they can 'learn' insights from that.

Furthermore you're thinking here doesn't even begin to explain multimodal models at all.

[ Parent | on HN ]
Re:Hallucination is inevitable: An innate limitation of larg... by DiogenesKynikos on Sunday February 25, 2024 @05:40PM (#39505665)
I just asked ChatGPT to make up a Chinese word for hungry+angry. It came up with a completely novel word that actually sounds okay: 饥怒. It then explained to me how it came up with the word.

You can't claim that that isn't understanding. It just strikes me that we've moved the goalposts into every more esoteric corners: sure, ChatGPT seems like it can have a real conversation, but can it do X extremely difficult task that I just thought up?

[ Parent | on HN ]