Re:Better and Faster Large Language Models via Multi-Token P...
by wantsanagent on Wednesday May 01, 2024 @11:33AM
(#40224733)
> "It is not designed to tell you the "most likely" text that follows,"
It is exactly designed to do that. A temperature of 0 this is what you are approximating. The crucial point though is that it is the most likely next word given the proceeding multi-token context, not just the previous token.
It is exactly designed to do that. A temperature of 0 this is what you are approximating. The crucial point though is that it is the most likely next word given the proceeding multi-token context, not just the previous token.