Re:Better and Faster Large Language Models via Multi-Token P...
by cgearhart on Wednesday May 01, 2024 @11:17AM
(#40224498)
What you’ve described is basically the problem with greedy sampling in the decoder. Many other local optimization sampling strategies exist (e.g., beam search) and there’s been a lot of work on more global sampling (e.g., speculative decoding).
[ Parent
| on HN ]
There are no comments on this comment yet.