News for hackers. Stuff that matters.
Re:Mercury 2: The fastest reasoning LLM, powered by diffusion by bigbuppo on Tuesday February 24, 2026 @09:13PM (#47146440)
Maybe make that intelligence per token per relative unit of hardware per watt. If you're burning 30 tons of coal to be 0.0000000001% better than the 5 tons of coal option because you're throwing more hardware at it, well, it's not much of a real improvement.
Re:Mercury 2: The fastest reasoning LLM, powered by diffusion by estsauver on Tuesday February 24, 2026 @10:44PM (#47147046)
I think the fast inference options have historically been only marginally more expensive then their slow cousins. There's a whole set of research about optimal efficiency, speed, and intelligence pareto curves. If you can deliver even an outdated low intelligence/old model at high efficiency, everyone will be interested. If you can deliver a model very fast, everyone will be interested. (If you can deliver a very smart model, everyone is obviously the most interested, but that's the free space.)

But to be clear, 1000 tokens/second is WAY better. Anthropic's Haiku serves at ~50 tokens per second.