Re:Better and Faster Large Language Models via Multi-Token P...
by sebzim4500 on Wednesday May 01, 2024 @09:26AM
(#40222877)
This is how they work and it's a real problem when doing prediction with low temperatures.
IIRC you see weird patterns in LLM outputs since "an" is often less likely than "a" so you end up with fewer nouns beginning with vowels than you would expect.
IIRC you see weird patterns in LLM outputs since "an" is often less likely than "a" so you end up with fewer nouns beginning with vowels than you would expect.