News for hackers. Stuff that matters.
Re:DeepSeek-R1 with Dynamic 1.58-bit Quantization by iamnotagenius on Tuesday January 28, 2025 @09:13AM (#42852488)
would be great to have dynamic quants of V3-non-R1 version, as for some tasks it is good enough. Also would be very interesting to see degradation with dynamic quants on small/medium size MoEs, such as older Deepseek models, Mixtrals, IBM tiny Granite MoE. Would be fun if Granite 1b MoE will still be functioning at 1.58bit.
Re:DeepSeek-R1 with Dynamic 1.58-bit Quantization by danielhanchen on Tuesday January 28, 2025 @04:32PM (#42858329)
Oh yes multiple people have asked me about this - I'll see what I can do :)