Re:Structured Output from LLMs (Without Reprompting!)
by kelseyfrog on Monday July 17, 2023 @10:51AM
(#36758970)
I mean even Jsonformer used LogitsWarper when generating numbers, but yes arbitrary grammars are infinitely more powerful.
A relevant PR:
https://github.com/ggerganov/llama.cpp/pull/1773
The plan is to support arbitrary grammar files to constrain tokens generation, similar to the grammar files here:
https://github.com/antlr/grammars-v4