Re:Show HN: LLMs can generate valid JSON 100% of the time
by potatoman22 on Monday August 14, 2023 @05:58PM
(#37127320)
They recently added logit biases, so that's a start.
[ Parent
| on HN ]
Re:Show HN: LLMs can generate valid JSON 100% of the time
by remilouf on Tuesday August 15, 2023 @12:30PM
(#37135968)
It's limited to 300 logit biases at a time. Knowing GPT4's vocabulary is ~100k tokens it's not nearly enough to get reliable guided generation. Although it could work in some cases, and another advantage of this work is that we can determine that before generating.