News for hackers. Stuff that matters.
Re:LLMs use a surprisingly simple mechanism to retrieve some... by dragonwriter on Friday March 29, 2024 @06:26PM (#39869666)
> I often point out to others that a transformer like gpt-4 operates wholly on numbers- it knows nothing of meaning in the real world- nothing at all

This is like saying a brain operates wholly on electrochemical states and knowns nothing about meaning in the real world, though; the mechanistic description is accurate, the cognitive conclusion attached to it is, at best, based on unsupported conjecture about the relation of mechanism to understanding.

Re:LLMs use a surprisingly simple mechanism to retrieve some... by richardatlarge on Friday March 29, 2024 @06:49PM (#39869865)
All the magic in the universe will not allow you to find anything about a human from looking solely at the neural structure of its brain, however sophisticated. The brain is a representation (genome) of lived experience (phenome). The transformer has only ever experienced alphanumeric data input. Second hand experience.