Re:Show HN: LLMs can generate valid JSON 100% of the time
by catlifeonmars on Monday August 14, 2023 @08:40PM
(#37128690)
* * *
Re:Show HN: LLMs can generate valid JSON 100% of the time
by MereInterest on Monday August 14, 2023 @08:44PM
(#37128720)
LLMs work by outputting a value for each token, then using those values to generate a probability distribution. Usually, this will be through a function like softmax [0], but there's nothing preventing you from doing some post-processing first. That post processing could be aware of the tokens that would be valid as the next token in a JSON format, and set the probabilities of all other tokens to zero. That way, even if the training data contains malformed JSON, the generator is still constrained to produce valid JSON.
Re:Show HN: LLMs can generate valid JSON 100% of the time
by RyEgswuCsn on Wednesday August 16, 2023 @08:28AM
(#37146041)
But that is not the LLM learning to produce valid JSON —— as some other commenters mentioned, you can get valid json without the LLM.
Sure it could be useful, but not really impressive.
What if the training data contains malformed JSON? There ought to be a non-zero chance of the LLM producing invalid JSON, no?