Re:Fixing Tool-Call Accuracy in vLLM
by Creamsicle47 on Wednesday October 28, 2026 @03:30PM
(#50279544)
Technically true, but this is a very inefficient optimization strategy and usually indicates some structural issue with your setup.
When you know enough about the JSON structure to understand how to set a decoding order, then you can simply break the problem into smaller parts so it's more manageable.
Otherwise, you have these solutions: - GEPA-style prompt optimization for simple agents - If working with large models, you can just have them do step-by-step problem solving & relying on their CoT for accuracy. - A small model can simply be trained to do what you want
And if those don't work, I would be extremely surprised.
The order of the output fields can actually be used to tune performance of the outputs. It can be helpful to force the LLM to reason about specific fields first if later fields benefit from seeing those earlier fields' output. You have to test it because some times the results are surprising in directions you didn't expect.
XGrammar has an option somewhere to allow fields to appear in any order but I don't know if vLLM exposes it. There are some tradeoffs and limitations for allowing fields to show up in any order because there is some complexity in schema parsing like that.