News for hackers. Stuff that matters.
Re:Extracting financial disclosure and police reports with O... by throwup238 on Monday October 14, 2024 @02:55PM (#41840651)
Structured output uses "constrained decoding" under the hood. They convert the JSON schema to a context free grammar so that when the model samples tokens, invalid tokens are masked to have a probability of zero. It's much less likely to go off the rails.

There are no comments on this comment yet.