News for hackers. Stuff that matters.
Re:OK, I can partly explain the LLM chess weirdness now by Der_Einzige on Friday November 22, 2024 @01:08PM (#42216047)
Correct - Dynamic grammar based/constrained sampling can be used to, at each time-step, force the model to only make valid moves (and you don't have to do it in the prompt like this article does!!!)

I have NO idea why no one seems to do this. It's a similar issue with LLM-as-judge evaluations. Often they are begging to be combined with grammar based/constrained/structured sampling. So much good stuff in LLM land isn't used for no good reason! There are several libraries for implementing this easily, outlines, guidance, lm-format-enforcer, and likely many more. You can even do it now with OpenAI!

Oobabooga text gen webUI literally has chess as one of it's candidate examples of grammar based sampling!!!

There are no comments on this comment yet.