Something is afoot in the land of Qwen

Re:Something is afoot in the land of Qwen by abhikul0 on Wednesday March 04, 2026 @12:34PM (#47250939)

Are you running it locally with llama.cpp? If so, is it working without any tweaking of the chat template? The tool calls fail for me when using the default chat template, however it seems to work a whole lot better with this: https://huggingface.co/Qwen/Qwen3.5-35B-A3B/discussions/9#69...

[ Parent | on HN ]

Re:Something is afoot in the land of Qwen by arcanemachiner on Wednesday March 04, 2026 @12:51PM (#47251195)

Have you tried the '--jinja' flag in llama-server?

[ Parent | on HN ]

Re:Something is afoot in the land of Qwen by abhikul0 on Wednesday March 04, 2026 @01:32PM (#47251771)

Yes, it fails too. I’m using the unsloth q4_km quant. Similarly fails with devstral2 small too, fixed that by using a similar template i found for it. Maybe it’s the quants that are broken, need to redownload I guess.

[ Parent | on HN ]

Re:Something is afoot in the land of Qwen by sosodev on Wednesday March 04, 2026 @08:36PM (#47256366)

I’ve been running it via llama-server with no issues. Running the latest Bartowski 6-bit quant

[ Parent | on HN ]

Re:Something is afoot in the land of Qwen by abhikul0 on Thursday March 05, 2026 @12:02AM (#47257778)

Thanks, i'll check his quants.

[ Parent | on HN ]

Re:Something is afoot in the land of Qwen by brightball on Wednesday March 04, 2026 @09:39PM (#47256820)

Bartowski? Like Chuck Bartowski from the TV show?

[ Parent | on HN ]

Re:Something is afoot in the land of Qwen by BoredomIsFun on Thursday March 05, 2026 @03:11AM (#47258994)

Different one. Bartowski is a minor celebrity in the local LLM world, together with Unsloth.

[ Parent | on HN ]

Re:Something is afoot in the land of Qwen by Balinares on Thursday March 05, 2026 @11:48AM (#47263967)

What's the selling point of these quants vs the Unsloth ones?

[ Parent | on HN ]

Re:Something is afoot in the land of Qwen by BoredomIsFun on Friday March 06, 2026 @04:34AM (#47272866)

Sometimes unsloth has broken ones for a particular model, sometimes no quants at all, and there is subtle difference in behavior.

[ Parent | on HN ]