News for hackers. Stuff that matters.
Re:Qwen/Qwen3.8-2.4T-A95B by Foobar8568 on Wednesday August 12, 2026 @03:13PM (#49277227)
I had several issues with unsloth gguf, even for models released a few months back like gemma 4, I have 0 confidence in their models, at this stage, I feel several uncensored are more reliable.
Re:Qwen/Qwen3.8-2.4T-A95B by segmondy on Wednesday August 12, 2026 @04:05PM (#49277872)
... because they are often the first to quant it. sometimes the actually model providers will release wrong chat templates or values in the model config which leads to bad quants. how would you know a quant is good if you don't make one? you don't. so they make it first, then they run a lot of tests, KD, perplexity, etc, they publish it. They take feedback from the community, then they update if needed. if you want to try it right now, you grab it else wait for a week or 2.
Re:Qwen/Qwen3.8-2.4T-A95B by Foobar8568 on Wednesday August 12, 2026 @04:16PM (#49277980)
Gemma4 was released a few weeks ago? The problems are still there. Today I started using another "provider" and the problems disappeared. Thanks but no.
Re:Qwen/Qwen3.8-2.4T-A95B by arcanemachiner on Wednesday August 12, 2026 @04:22PM (#49278053)
They are very responsive, and would probably be happy to help you fix your issues.
Re:Qwen/Qwen3.8-2.4T-A95B by danielhanchen on Wednesday August 12, 2026 @06:39PM (#49279552)
Hey yes - if you could describe what the issues are - we will gladly fix them!
Re:Qwen/Qwen3.8-2.4T-A95B by Foobar8568 on Thursday August 13, 2026 @12:58PM (#49288795)
Mea-culpa, dry-multiplier generated crap and even more so on Gemma4.
Re:Qwen/Qwen3.8-2.4T-A95B by danielhanchen on Friday August 14, 2026 @02:12AM (#49295255)
Ok no worries - if there are any future issues - feel free to message / make a HF issue - we'll fix promptly!

Also note its best to follow Gemma4's official sampling params since they evaled with it - dry multiplier sometimes works, but it actually screws up reasoning sometimes

Re:Qwen/Qwen3.8-2.4T-A95B by segmondy on Wednesday August 12, 2026 @06:39PM (#49279565)
I've 0 issues with gemma4 and I downloaded it early.
Re:Qwen/Qwen3.8-2.4T-A95B by LeBit on Wednesday August 12, 2026 @09:07PM (#49280619)
Same. Used the 12B, 26B A4B and 31B.

No issues with llama.cpp.

Re:Qwen/Qwen3.8-2.4T-A95B by jokethrowaway on Wednesday August 12, 2026 @05:52PM (#49279091)
I've used their gemma 4 quants since when they were not still working in llama.cpp and ik-llama.cpp and I don't remember any problems

They are the most reliable in my experience, but if you have alternatives you trust I'd love to know

Re:Qwen/Qwen3.8-2.4T-A95B by danielhanchen on Wednesday August 12, 2026 @06:38PM (#49279544)
Hey sorry what are the problems that you're experiencing - we're more than happy to help fix them!
Re:Qwen/Qwen3.8-2.4T-A95B by LeBit on Wednesday August 12, 2026 @09:05PM (#49280610)
Thank you for what you are doing.
Re:Qwen/Qwen3.8-2.4T-A95B by danielhanchen on Wednesday August 12, 2026 @10:54PM (#49281282)
Thanks for the support and to the community!
Re:Qwen/Qwen3.8-2.4T-A95B by chlorion on Wednesday August 12, 2026 @10:21PM (#49281117)
Huh I have had great luck with unsloth quants so far. What issues are you having?
Re:Qwen/Qwen3.8-2.4T-A95B by SwellJoe on Thursday August 13, 2026 @12:26AM (#49281788)
Counterpoint: I've been using the Unsloth Gemma 4 quants extensively since very soon after release (on ROCm and Apple Silicon, I don't have any Nvidia hardware big enough), pretty much every quantization down to 4 bits (the QAT is the business, indistinguishable from the full-fat version, runs great on a slightly chonky desktop or laptop), and I haven't had any issues. The reason I use Gemma 4 so much often comes down to how reliable it is; when I want to experiment with llama.cpp settings, MTP, n-gram, etc. it's my go-to because I know there isn't anything wrong with the model or the quantizations that could interfere with the experiment.

It did take a little while for Unsloth to update the Laguna S 2.1 quants to fix the yarn_attn_factor, and so it was a bit frustrating getting that quantization running right, but almost always, I pick the unsloth quantization if there is one. (Still waiting/hoping for a Ling 3.0 Flash.)

Re:Qwen/Qwen3.8-2.4T-A95B by danielhanchen on Thursday August 13, 2026 @03:58AM (#49282955)
We will investigate Ling!
Re:Qwen/Qwen3.8-2.4T-A95B by BonerWiener on Thursday August 13, 2026 @03:49AM (#49282896)
> I feel several uncensored are more reliable.

I have had the same experience with gemma 4 on same tasks being refused. But this is when working with cyber offensive tasks and the like. It excels in coding and is very fast on consumer hardware. So I would say use the right tool for the right task.

What uncensored models can you recommend?

Re:Qwen/Qwen3.8-2.4T-A95B by jacquesm on Thursday August 13, 2026 @06:14AM (#49283813)
Without substantiating what issues you have with unsloth gguf files you are just adding noise, no signal.