Running Qwen3.8 at 255 tok/s on a single 5090

Re:Running Qwen3.8 at 255 tok/s on a single 5090 by gregojaca on Wednesday October 14, 2026 @10:25AM (#50088245)

What's the prefill speed and time to first token? Did you see any slowdown at concurrency for the prefill or decode?

[ Parent | on HN ]

Re:Running Qwen3.8 at 255 tok/s on a single 5090 by MozerJ on Wednesday October 14, 2026 @01:01PM (#50090943)

TTFT is around 180ms, prefill speed is ~2.7k tok/sec. Yes, both prefill and decode slows down as concurrency increases (it's not linear). This 255 tok/s is at conc = 1.

[ Parent | on HN ]