News for hackers. Stuff that matters.
Re:Running GPT-OSS-120B at 500 tokens per second on Nvidia GPUs by wkat4242 on Thursday August 07, 2025 @05:05AM (#44822256)
Wow nice!! That's a really good deal for that much hardware.

How many tokens/s do you get for DeepSeek-R1?

Re:Running GPT-OSS-120B at 500 tokens per second on Nvidia GPUs by DrPhish on Thursday August 07, 2025 @10:29AM (#44824925)
Thanks, it was a bit of a gamble at the time (lots of dodgy ebay parts), but it paid off.

R1 starts at about 10t/s on an empty context but quickly falls off. I'd say the majority of my tokens are generating around 6t/s.

Some of the other big MoE models can be quite a bit faster.

I'm mostly using QwenCoder 480b at Q8 these days for 9t/s average. I've found I get better real-world results out of it than K2, R1 or GLM4.5.