News for hackers. Stuff that matters.
Re:Structured Outputs in LLMs by nl on Tuesday September 23, 2025 @10:14AM (#45347328)
Most API providers (Together, Fireworks etc) don't build their own models.
Re:Structured Outputs in LLMs by 2THFairy on Tuesday September 23, 2025 @12:15PM (#45349070)
You don't need a new model. The trick of the technique is that you only change how tokens are sampled; Zero out the probability of every token that would be illegal under the grammar or other constraints.

All you need for that is an inference API that gives you the full output vector, which is trivial for any model you run on your own hardware.

Re:Structured Outputs in LLMs by ijk on Tuesday September 23, 2025 @10:19AM (#45347396)
Though Fireworks is one of the few providers that supports structured generation.