News for hackers. Stuff that matters.
Re:Fixing Tool-Call Accuracy in vLLM by Aurornis on Monday October 26, 2026 @11:32AM (#50245479)
Constrained decoding has been in vLLM for a long time, and constrained tool call decoding has been in vLLM for a while, too.

The blog post is rather confusing because until you understand the entire situation it appears to imply that they fixed vLLM (what the title implies) but really they just needed to configure it correctly.

To be fair, the course of constrained tool calling configuration in vLLM changed over time in ways that changed behavior, which is the real problem. The Ubicloud team deserves a lot of credit for having benchmarks in place to track performance and catch regressions for investigations. Some of the providers you find on OpenRouter have inexplicably bad performance relative to others and clearly do not care this much.

However, if this article left you confused because constrained decoding and constrained tool call schema have been features in vLLM for a long time, jump to the end where they show what configuration changes they made.

Re:Fixing Tool-Call Accuracy in vLLM by jesdo on Monday October 26, 2026 @11:42AM (#50245660)
Overall, a fair assessment. I do want to place a footnote here that we did fix vLLM in the sense that its correct behavior for Kimi required code changes: it was impossible to fix it in the configuration. Later, the vLLM team fixed this problem upstream. Now, with the latest version of vLLM, the problem we describe can be addressed by setting the `--tool-strict-level` flag to `parameter`.
Re:Fixing Tool-Call Accuracy in vLLM by wredcoll on Tuesday October 27, 2026 @01:10AM (#50254922)
If you work for ubicloud, I have a question: do you sell api access to kimi/etc, and if so, where is that listed??
Re:Fixing Tool-Call Accuracy in vLLM by jesdo on Tuesday October 27, 2026 @11:25AM (#50260772)
We currently focus on serving DeepSeek models on dedicated clusters for large customers. We do not offer this as a publicly available service at the moment, but we may consider doing so in the future.
Re:Fixing Tool-Call Accuracy in vLLM by jacquesm on Monday October 26, 2026 @11:35AM (#50245521)
In part this is because VLLM is a confusing mess. It takes a large investment in time to get it to perform well (and when it does it is really good). But it is fragile and over complicated.
Re:Fixing Tool-Call Accuracy in vLLM by svcrunch on Monday October 26, 2026 @11:45AM (#50245707)
> The blog post is rather confusing because until you understand the entire situation it appears to imply that they fixed vLLM (what the title implies) but really they just needed to configure it correctly.

Just as a point of feedback for the authors, I enjoyed the article, and did not suffer the confusion pointed out above.

Re:Fixing Tool-Call Accuracy in vLLM by enoch2090 on Monday October 26, 2026 @10:16PM (#50253951)
Should have just wrote "Fixing Tool-Call Accuracy with vLLM" instead of "Fixing Tool-Call Accuracy in vLLM"