News for hackers. Stuff that matters.
Re:Qwen3.6-35B-A3B: Agentic Coding Power, Now Open to All by danielhanchen on Thursday April 16, 2026 @11:46AM (#47795073)
We re-uploaded Gemma4 4 times - 3 times were due to 20 llama.cpp bug fixes, which we helped solve some as well. The 4th is an official Gemma chat template improvement from Google themselves, so these are out of our hands. All providers had to re-fix their uploads, so not just us.

For MiniMax 2.7 - there were NaNs, but it wasn't just ours - all quant providers had it - we identified 38% of bartowski's had NaNs. Ours was 22%. We identified a fix, and have already fixed ours see https://www.reddit.com/r/LocalLLaMA/comments/1slk4di/minimax.... Bartowski has not, but is working on it. We share our investigations always.

For Qwen3.5 - we shared our 7TB research artifacts showing which layers not to quantize - all provider's quants were not optimal, not broken - ssm_out and ssm_* tensors were the issue - we're now the best in terms of KLD and disk space - see https://www.reddit.com/r/LocalLLaMA/comments/1rgel19/new_qwe...

On other fixes, we also fixed bugs in many OSS models like Gemma 1, Gemma 3, Llama chat template fixes, Mistral, and many more.

It might seem these issues are due to us, but it's because we publicize them and tell people to update. 95% of them are not related to us, but as good open source stewards, we should update everyone.

Re:Qwen3.6-35B-A3B: Agentic Coding Power, Now Open to All by sowbug on Thursday April 16, 2026 @11:57AM (#47795273)
Please publish sha256sums of the merged GGUFs in the model descriptions. Otherwise it's hard to tell if the version we have is the latest.
Re:Qwen3.6-35B-A3B: Agentic Coding Power, Now Open to All by danielhanchen on Thursday April 16, 2026 @12:01PM (#47795358)
Yep we can do that probs add a table - in general be post in discussions of model pages - for eg https://huggingface.co/unsloth/MiniMax-M2.7-GGUF/discussions...

HF also provides SHA256 for eg https://huggingface.co/unsloth/MiniMax-M2.7-GGUF/blob/main/U... is 92986e39a0c0b5f12c2c9b6a811dad59e3317caaf1b7ad5c7f0d7d12abc4a6e8

But agreed it's probs better to place them in a table

Re:Qwen3.6-35B-A3B: Agentic Coding Power, Now Open to All by sowbug on Thursday April 16, 2026 @12:12PM (#47795538)
Thanks! I know about HF's chunk checksums, but HF doesn't publish (or possibly even know) the merged checksums.
Re:Qwen3.6-35B-A3B: Agentic Coding Power, Now Open to All by danielhanchen on Thursday April 16, 2026 @12:56PM (#47796233)
Oh for multi files? Hmm ok let me check that out
Re:Qwen3.6-35B-A3B: Agentic Coding Power, Now Open to All by zargon on Thursday April 16, 2026 @01:56PM (#47797097)
Why do you merge the GGUFs? The 50 GB files are more manageable (IMO) and you can verify checksums as you say.
Re:Qwen3.6-35B-A3B: Agentic Coding Power, Now Open to All by sowbug on Thursday April 16, 2026 @03:19PM (#47798194)
I admit it's a habit that's probably weeks out of date. Earlier engines barfed on split GGUFs, but support is a lot better now. Frontends didn't always infer the model name correctly from the first chunk's filename, but once llama.cpp added the models.ini feature, that objection went away.

The purist in me feels the 50GB chunks are a temporary artifact of Hugging Face's uploading requirements, and the authoritative model file should be the merged one. I am unable to articulate any practical reason why this matters.

Re:Qwen3.6-35B-A3B: Agentic Coding Power, Now Open to All by dist-epoch on Thursday April 16, 2026 @12:18PM (#47795658)
What do you think about creating a tool which can just patch the template embedded in the .gguf file instead of forcing a re-download? The whole file hash can be checked afterwards.
Re:Qwen3.6-35B-A3B: Agentic Coding Power, Now Open to All by danielhanchen on Thursday April 16, 2026 @12:58PM (#47796262)
Sadly it's not always chat template fixes :( But yes we now split the first shard as pure metadata (10MB) for huge models - these include the chat template etc - so you only need to download that.

For serious fixes, sadly we have to re-compute imatrix since the activation patterns have changed - this sadly makes the entire quant change a lot, hence you have to re-download :(

Re:Qwen3.6-35B-A3B: Agentic Coding Power, Now Open to All by evilduck on Thursday April 16, 2026 @12:36PM (#47795939)
I just wanted to express gratitude to you guys, you do great work. However, it is a little annoying to have to redownload big models though and keeping up with the AI news and community sentiment is a full time job. I wish there was some mechanism somewhere (on your site or Huggingface or something) for displaying feedback or confidence in a model being "ready for general use" before kicking off 100+ GB model downloads.
Re:Qwen3.6-35B-A3B: Agentic Coding Power, Now Open to All by danielhanchen on Thursday April 16, 2026 @12:55PM (#47796224)
Hey thanks - yes agreed - for now we do:

1. Split metadata into shard 0 for huge models so 10B is for chat template fixes - however sometimes fixes cause a recalculation of the imatrix, which means all quants have to be re-made

2. Add HF discussion posts on each model talking about what changed, and on our Reddit and Twitter

3. Hugging Face XET now has de-duplication downloading of shards, so generally redownloading 100GB models again should be much faster - it chunks 100GB into small chunks and hashes them, and only downloads the shards which have changed

Re:Qwen3.6-35B-A3B: Agentic Coding Power, Now Open to All by evilduck on Thursday April 16, 2026 @03:40PM (#47798466)
Ah thanks, I wasn't aware of #3, that should be a huge boon.
Re:Qwen3.6-35B-A3B: Agentic Coding Power, Now Open to All by danielhanchen on Friday April 17, 2026 @04:25AM (#47803698)
Oh yes! This only applies if one uses hf download / snapshot_download - other normal download methods sadly won't have XET
Re:Qwen3.6-35B-A3B: Agentic Coding Power, Now Open to All by ssrshh on Thursday April 16, 2026 @09:37PM (#47801638)
If you would know - is this also why LM Studio and Ollama model downloads often fail with a signature mismatch error?
Re:Qwen3.6-35B-A3B: Agentic Coding Power, Now Open to All by danielhanchen on Friday April 17, 2026 @04:25AM (#47803693)
Probably yes
Re:Qwen3.6-35B-A3B: Agentic Coding Power, Now Open to All by CamperBob2 on Thursday April 16, 2026 @01:42PM (#47796909)
Best policy is to just wait a couple of weeks after a major model is released. It's frustrating to have to re-download tens or hundreds of GB every few days, but the quant producers have no choice but to release early and often if they want to maintain their reputation.

Ideally the labs releasing the open models would work with Unsloth and the llama.cpp maintainers in advance to work out the bugs up front. That does sometimes happen, but not always.

Re:Qwen3.6-35B-A3B: Agentic Coding Power, Now Open to All by danielhanchen on Thursday April 16, 2026 @02:04PM (#47797225)
Yep agreed at least 1 week is a good idea :)

We do get early access to nearly all models, and we do find the most pressing issues sometimes. But sadly some issues are really hard to find and diagnose :(

Re:Qwen3.6-35B-A3B: Agentic Coding Power, Now Open to All by magicalhippo on Thursday April 16, 2026 @03:31PM (#47798336)
Appreciate the work of your team very much.

Though chat templates seem like they need a better solution. So many issues, seems quite fragile.

Re:Qwen3.6-35B-A3B: Agentic Coding Power, Now Open to All by danielhanchen on Friday April 17, 2026 @04:27AM (#47803705)
Thank you! Agreed on chat template issue
Re:Qwen3.6-35B-A3B: Agentic Coding Power, Now Open to All by solomatov on Thursday April 16, 2026 @06:58PM (#47800591)
Just curious, the fixes are not about weights but about templates, am I right?
Re:Qwen3.6-35B-A3B: Agentic Coding Power, Now Open to All by danielhanchen on Friday April 17, 2026 @04:27AM (#47803707)
Yes so chat templates and the actual implementations