Re:Unsloth Qwen3.8-27B Quantizations – Dynamic v3.1 Update
by DiabloD3 on Saturday November 14, 2026 @05:57AM
(#50495973)
IMO Byteshape is kind of the winner here if you're only looking at 24GB and down targets.
Continually compressing the model down with more computationally complex quantization schemes crosses from memory bandwidth being the limit into compute is the limit, Byteshape tries to keep the models performant while also using techniques appropriately.
Like, for me, their biggest version saves enough space that I can actually fit the dflash2 kernel without digging into context, yet score virtually identical to Unsloth's 3.0 Q4_K_M.
Thats like going from, say, 15-20 tok/s with MTP to 50+ with DFlash2.
Continually compressing the model down with more computationally complex quantization schemes crosses from memory bandwidth being the limit into compute is the limit, Byteshape tries to keep the models performant while also using techniques appropriately.
Like, for me, their biggest version saves enough space that I can actually fit the dflash2 kernel without digging into context, yet score virtually identical to Unsloth's 3.0 Q4_K_M.
Thats like going from, say, 15-20 tok/s with MTP to 50+ with DFlash2.