Re:Qwen 3.8-Flash-Next releasing tomorrow (125B a6B)
by decide1000 on Tuesday August 25, 2026 @05:27PM
(#49440904)
On the DGX I get 44.5 tokens per second (NVFP4). With 8 concurrent it's 241 t/s total.
I am using the PrismaAQUA
standard 9.7 t/s
+ Dflash2 30 t/s
+ torch-compile 37 t/s
c8 = 177 t/s
Re:Qwen 3.8-Flash-Next releasing tomorrow (125B a6B)
by SwellJoe on Tuesday August 25, 2026 @05:33PM
(#49440969)
What model? Also, I don't know what "the PrismaAQUA" means, ddg thinks it's a CPAP machine, which seems unlikely to help with inference performance.
Also, 4-bit has measurable intelligence loss. Sometimes worth it, but, at this size models are barely smart enough at 8 or 6.
Re:Qwen 3.8-Flash-Next releasing tomorrow (125B a6B)
by decide1000 on Tuesday August 25, 2026 @07:04PM
(#49441888)
Qwen3.8-27B-PrismaAQUA-5.5bit-vllm
The output quality is higher. It's held at full precision (not quantized).
A model similar in size to Laguna S 2.1, but with only 6B active parameters, should be a notable amount faster, so I would imagine 25-30 t/s would be a reasonable guess for where Qwen 3.8 Flash Next will land.
DFlash2 might improve all these numbers. It wasn't available last I was testing new models on the Strix Halo; I've only used MTP (which doesn't generally improve MoE models, but I believe DFlash2 can).
Given software improvements, I'm hopeful an MoE in this size range will be the sweet spot that pushes past 40 t/s and is also smart enough for real work. Qwen 3.8 27B is finally a self-hostable model that's smart enough, but it thinks so hard it still isn't really useful for agentic interactive use.