News for hackers. Stuff that matters.
Re:3Blue1Brown: Visualizing Attention, a Transformer's Heart by astrange on Monday April 15, 2024 @02:41PM (#40044158)
LLMs don't work on words, they work on sequences of subword tokens. "It doesn't actually do anything" is a common explanation that's clearly a form of cope, because you can't even explain why it can form complete words, let alone complete sentences.

There are no comments on this comment yet.