News for hackers. Stuff that matters.
Re:Better and Faster Large Language Models via Multi-Token P... by wantsanagent on Wednesday May 01, 2024 @11:33AM (#40224733)
> "It is not designed to tell you the "most likely" text that follows,"

It is exactly designed to do that. A temperature of 0 this is what you are approximating. The crucial point though is that it is the most likely next word given the proceeding multi-token context, not just the previous token.

Re:Better and Faster Large Language Models via Multi-Token P... by elcomet on Thursday May 02, 2024 @03:54PM (#40240717)
It's not designed to output the most likely séquence. It's designed to output the most likely next token. That's quite different.