News for hackers. Stuff that matters.
Re:Introducing TypeChat from Microsoft by Scaevolus on Saturday July 22, 2023 @12:38PM (#36827641)
Llama.cpp recently added grammar based sampling, which constraints token selection to follow a rigid format like you describe.

https://github.com/ggerganov/llama.cpp/pull/1773

There are no comments on this comment yet.