Re:Σ-GPTs: A New Approach to Autoregressive Models
by zakkor1 on Saturday June 08, 2024 @02:11AM
(#40615642)
> Fix curly brace tokens to the beginning
Regular LLMs can already do this, by prefilling the start of the assistant's response.
But there is actually something even better: you can constrain the LLM's output to a specific grammar (like JSON), so it'll only be able to answer with syntactically valid JSON.
Re:Σ-GPTs: A New Approach to Autoregressive Models
by vessenes on Saturday June 08, 2024 @03:31PM
(#40619708)
Yes. And you can have a grammar parser only select from valid tokens in a randomized distribution. But, this feels much more sophisticated to me, especially if you can mix specific token-based grammar requirements with other instructions during the token selection phase.
A number of things that are relatively hard for sequential LLMs are easy here. Want json? Fix curly brace tokens to the beginning and end.
Want a specific token-length explanation of an answer? Write a short answer, post-pend it, and infill.
Want a higher-density answer to something? Add a density assessment section to your generated text, a space for the LLM to score info-density, and generate looking for a high density score.
I would guess there's a lot here to be experimented with. It would be nice to get an 8b parameter model with reasonable number of tokens (x3 based on the paper, sadly) through it.