Re:Cerebras Launches Qwen3-235B, Achieving 1,500 Tokens per ...
by vlovich123 on Wednesday July 23, 2025 @02:08PM
(#44662193)
What quantization formats are these? All the OSS ones from GGML apply a uniform quantization
[ Parent
| on HN ]
Re:Cerebras Launches Qwen3-235B, Achieving 1,500 Tokens per ...
by int_19h on Wednesday July 23, 2025 @03:07PM
(#44662801)
GGML hasn't been a thing for some time, and GGUF (its successor) has features such as "importance matrix" quantization that is all about quantizing adaptively. Then there's all the stuff that Unsloth does, e.g.: https://unsloth.ai/blog/dynamic-v2
Re:Cerebras Launches Qwen3-235B, Achieving 1,500 Tokens per ...
by smcleod on Wednesday July 23, 2025 @04:52PM
(#44663919)
No they don't. GGML is non-uniform. Each layer gets its own level of quantisation.
Re:Cerebras Launches Qwen3-235B, Achieving 1,500 Tokens per ...
by smcleod on Thursday July 24, 2025 @07:04PM
(#44677380)
Have a watch of https://youtu.be/vW30o4U9BFE