Hm, that seems strangely slow. Qwen3.8-27B/DaisyRose-27B Q4 currently run with ~66 t/s and ~800 t/s prefill with 3 slots and about 190k kv cache. It performs really well so far.
Yeah I've had 27B (dense model) up to similar speeds on this setup as well. I'll hold off to compare my speeds to others, maybe it's too early or it's particular to my setup.
I'm getting 700 tok/s prefill and 52tok/s generation on a 5090 RTX, not bad
5090 RTX is like 1800GB/s? I bet you were seeing much higher speeds on Qwen 3.8 27B right?
edit: found some oudated mesa/radv/shaderc/glslang packages to update, and up to 34 tok/s now. MORE MORE