Re:Σ-GPTs: A New Approach to Autoregressive Models
by vessenes on Saturday June 08, 2024 @03:31PM
(#40619708)
Yes. And you can have a grammar parser only select from valid tokens in a randomized distribution. But, this feels much more sophisticated to me, especially if you can mix specific token-based grammar requirements with other instructions during the token selection phase.
Regular LLMs can already do this, by prefilling the start of the assistant's response.
But there is actually something even better: you can constrain the LLM's output to a specific grammar (like JSON), so it'll only be able to answer with syntactically valid JSON.