News for hackers. Stuff that matters.
Re:Better and Faster Large Language Models via Multi-Token P... by faabian on Wednesday May 01, 2024 @08:37AM (#40222289)
Language models factor the joint probability p(y, x) as p(y, x) = p(y|x) p(x) which is exact. I.e. if you train a language model on your distribution and sample with temperature 1, you will get the exact same distribution out. If you sample at lower temperature or even greedily, evidently, you will get other distributions.

There are no comments on this comment yet.