News for hackers. Stuff that matters.
Re:Cerebras Launches Qwen3-235B, Achieving 1,500 Tokens per ... by vlovich123 on Wednesday July 23, 2025 @02:08PM (#44662193)
What quantization formats are these? All the OSS ones from GGML apply a uniform quantization
Re:Cerebras Launches Qwen3-235B, Achieving 1,500 Tokens per ... by int_19h on Wednesday July 23, 2025 @03:07PM (#44662801)
GGML hasn't been a thing for some time, and GGUF (its successor) has features such as "importance matrix" quantization that is all about quantizing adaptively. Then there's all the stuff that Unsloth does, e.g.: https://unsloth.ai/blog/dynamic-v2
Re:Cerebras Launches Qwen3-235B, Achieving 1,500 Tokens per ... by smcleod on Wednesday July 23, 2025 @04:52PM (#44663919)
No they don't. GGML is non-uniform. Each layer gets its own level of quantisation.
Re:Cerebras Launches Qwen3-235B, Achieving 1,500 Tokens per ... by smcleod on Thursday July 24, 2025 @07:04PM (#44677380)