News for hackers. Stuff that matters.
Re:Qwen3-Next by EnPissant on Friday September 12, 2025 @01:28PM (#45224486)
The slowdown is far more than 15% for token generation. Token generation is mostly bottlenecked by memory bandwidth. Dual channel DDR5-6000 has 96GB/s and A rtx 5090 has 1.8TB/s. See my other comment when I show 5x slowdown in token generation by moving just the experts to the CPU.
Re:Qwen3-Next by DiabloD3 on Saturday September 13, 2025 @02:27AM (#45229800)
I suggest figuring out what your configuration problem is.

Which llama.cpp flags are you using, because I am absolutely not having the same bug you are.

Re:Qwen3-Next by EnPissant on Saturday September 13, 2025 @02:44AM (#45229879)
It's not a bug. It's the reality of token generation. It's bottlenecked by memory bandwidth.

Please publish your own benchmarks proving me wrong.

Re:Qwen3-Next by DiabloD3 on Sunday September 14, 2025 @08:25AM (#45239297)
I cannot reproduce your bug on AMD. I'm going to have to conclude this is a vendor issue.