News for hackers. Stuff that matters.

A comment on Running Qwen3.8 at 255 tok/s on a single 5090. Parent comment

Re:Running Qwen3.8 at 255 tok/s on a single 5090 by MozerJ on Wednesday October 14, 2026 @01:01PM (#50090943)
TTFT is around 180ms, prefill speed is ~2.7k tok/sec. Yes, both prefill and decode slows down as concurrency increases (it's not linear). This 255 tok/s is at conc = 1.
[ Parent | on HN ]

There are no comments on this comment yet.