News for hackers. Stuff that matters.
Re:Mercury 2: The fastest reasoning LLM, powered by diffusion by lostmsu on Tuesday February 24, 2026 @10:41PM (#47147033)
Regular models are very fast if you do batch inference. GPT-OSS 20B gets close to 2k tok/s on a single 3090 at bs=64 (might be misremembering details here).
Re:Mercury 2: The fastest reasoning LLM, powered by diffusion by rahimnathwani on Wednesday February 25, 2026 @01:59AM (#47148258)
Right but everyone else is talking about latency, not throughput.