Re:LLMs use a surprisingly simple mechanism to retrieve some...
by richardatlarge on Friday March 29, 2024 @06:49PM
(#39869865)
All the magic in the universe will not allow you to find anything about a human from looking solely at the neural structure of its brain, however sophisticated. The brain is a representation (genome) of lived experience (phenome). The transformer has only ever experienced alphanumeric data input. Second hand experience.
This is like saying a brain operates wholly on electrochemical states and knowns nothing about meaning in the real world, though; the mechanistic description is accurate, the cognitive conclusion attached to it is, at best, based on unsupported conjecture about the relation of mechanism to understanding.