Re:Apple Announces Foundation Models and Containerization fr...
by lqstuart on Tuesday June 10, 2025 @08:13PM
(#44242928)
Haha yeah. I’ve seen you mention the llama cpp wrapper elsewhere, it sounds cool! I’ve worked enough with vLLM and sglang to get angry at xgrammar, which I believe has some common ancestry with the GGML stack (GBNF if I’m not mistaken, which I may be). The constrained decoding part is as simple as you’d expect, just applies a bitmask to the logprobs during the “logit processing” and continuing as normal.
[ Parent
| on HN ]
There are no comments on this comment yet.