News for hackers. Stuff that matters.
Re:Qwen3-235B-A22B-Thinking-2507 by lostmsu on Friday July 25, 2025 @12:44PM (#44685191)
How usable are 1 and less than 1 bit quants?
Re:Qwen3-235B-A22B-Thinking-2507 by danielhanchen on Friday July 25, 2025 @04:20PM (#44687954)
Oh 1bit reminder isn't 1bit, but a mixture of 1 to 8bit! I would still use 2bit dynamic - 1bit dynamic sometimes can go into weird repetitive loops, but it still does produce reasonable output.

Larger models with 1bit do better - for eg 480B Coder 1bit actually does very well!