Re:Something weird is happening with LLMs and chess
by int_19h on Friday November 15, 2024 @02:26PM
(#42150017)
A grammar is really just a special case of the more general issue of how to pick a single token given the probabilities that the model spits out for every possible one. In that sense, filters like temperature / top_p / top_k are already hacks that "do the work" (since always taking the most likely predicted token does not give good results in practice), and grammars are just a more complicated way to make such decisions.
[ Parent
| on HN ]
There are no comments on this comment yet.