News for hackers. Stuff that matters.
Re:Qwen 3.8-Flash-Next releasing tomorrow (125B a6B) by SwellJoe on Tuesday August 25, 2026 @12:03PM (#49436325)
The Strix Halo and DGX Spark are pretty danged slow, relatively speaking. I don't recall exact numbers, but with MoE models in this size ballpark (Laguna S 2.1), I seem to recall I was seeing about 20-25 t/s with a big context, which is close to usable. Qwen 3.8 27B crawls on this hardware, though, at 10-16 t/s, definitely not comfortable for interactive use. (Though this makes it seem like you can cook pretty good with a 4-bit ROCmFP4 quantization: https://github.com/julianmb/q38rocm the model does get notably dumber below six bits.)

A model similar in size to Laguna S 2.1, but with only 6B active parameters, should be a notable amount faster, so I would imagine 25-30 t/s would be a reasonable guess for where Qwen 3.8 Flash Next will land.

DFlash2 might improve all these numbers. It wasn't available last I was testing new models on the Strix Halo; I've only used MTP (which doesn't generally improve MoE models, but I believe DFlash2 can).

Given software improvements, I'm hopeful an MoE in this size range will be the sweet spot that pushes past 40 t/s and is also smart enough for real work. Qwen 3.8 27B is finally a self-hostable model that's smart enough, but it thinks so hard it still isn't really useful for agentic interactive use.

Re:Qwen 3.8-Flash-Next releasing tomorrow (125B a6B) by decide1000 on Tuesday August 25, 2026 @05:27PM (#49440904)
On the DGX I get 44.5 tokens per second (NVFP4). With 8 concurrent it's 241 t/s total.

I am using the PrismaAQUA

standard 9.7 t/s

+ Dflash2 30 t/s

+ torch-compile 37 t/s

c8 = 177 t/s

Re:Qwen 3.8-Flash-Next releasing tomorrow (125B a6B) by SwellJoe on Tuesday August 25, 2026 @05:33PM (#49440969)
What model? Also, I don't know what "the PrismaAQUA" means, ddg thinks it's a CPAP machine, which seems unlikely to help with inference performance.

Also, 4-bit has measurable intelligence loss. Sometimes worth it, but, at this size models are barely smart enough at 8 or 6.

Re:Qwen 3.8-Flash-Next releasing tomorrow (125B a6B) by decide1000 on Tuesday August 25, 2026 @07:04PM (#49441888)
Qwen3.8-27B-PrismaAQUA-5.5bit-vllm

The output quality is higher. It's held at full precision (not quantized).