News for hackers. Stuff that matters.
Re:Better and Faster Large Language Models via Multi-Token P... by tmoertel on Wednesday May 01, 2024 @10:43AM (#40223965)
> And a final note, predicting one token at a time is what we are doing as humans when we speak...

I wouldn't be surprised if we could predict token groups. When speaking off the cuff, people often rely on well-worn phrases and cliches.

Re:Better and Faster Large Language Models via Multi-Token P... by abakker on Wednesday May 01, 2024 @11:35AM (#40224765)
Indeed. An interesting reference to this is the work Millman Parry did to describe the key phrases in the Odyssey and the queues they gave to help someone memorize the poem.

Also, this is maybe a semantic point, but, I am not predicting any words I speak. Not in a statistical sense. I have intent behind my words, which means I have an abstraction of meaning that I want to convey and I assemble the correct words to do that. no part of that is "predictive"

Re:Better and Faster Large Language Models via Multi-Token P... by thwarted on Wednesday May 01, 2024 @11:42AM (#40224863)
> describe the key phrases in the Odyssey and the queues they gave to help someone memorize the poem.

Queues of words give cues to help memorize.

Re:Better and Faster Large Language Models via Multi-Token P... by abakker on Wednesday May 01, 2024 @11:54AM (#40225041)
lol. Hurray for speech to text, I guess.
Re:Better and Faster Large Language Models via Multi-Token P... by ivalm on Wednesday May 01, 2024 @11:32AM (#40224727)
I think that corresponds to well worn phrases being a “single token”