Hallucination is inevitable: An innate limitation of large language models

Re:Hallucination is inevitable: An innate limitation of larg... by pixl97 on Sunday February 25, 2024 @05:20PM (#39505495)

>You don’t need proper cognition to identify that the answer is not stored in source data.

Uh, what?

So lets imagine you have an LLM that knows everything, except you withhold the data that you can put peanut butter on toast. Toast + Peanut butter = does not exist in data set. So what exactly do you expect the LLM to say when someone asks "Can you put peanut butter on toast?".

I would expect an intelligent agent to 'think' Peanut butter = spreadable food, toast = hard food substrate, so yea, they should work instead of the useless answer of I don't know.

Everything that does not exist in nature is made up by humans, the question is not "is it made up" the question is "does it work"

[ Parent | on HN ]

Re:Hallucination is inevitable: An innate limitation of larg... by jncfhnb on Monday February 26, 2024 @03:39AM (#39508881)

This behavior you’re describing is trainable either way.

Tuned LLMs are not simple most likely token models. They are most likely token given a general overarching strategy for contextualizing future tokens model.

Which can be conservative or imaginative.

[ Parent | on HN ]