News for hackers. Stuff that matters.
Re:Mercury 2: The fastest reasoning LLM, powered by diffusion by cjbarber on Tuesday February 24, 2026 @07:28PM (#47145617)
It could be interesting to do the metric of intelligence per second.

ie intelligence per token, and then tokens per second

My current feel is that if Sonnet 4.6 was 5x faster than Opus 4.6, I'd be primarily using Sonnet 4.6. But that wasn't true for me with prior model generations, in those generations the Sonnet class models didn't feel good enough compared to the Opus class models. And it might shift again when I'm doing things that feel more intelligence bottlenecked.

But fast responses have an advantage of their own, they give you faster iteration. Kind of like how I used to like OpenAI Deep Research, but then switched to o3-thinking with web search enabled after that came out because it was 80% of the thoroughness with 20% of the time, which tended to be better overall.

Re:Mercury 2: The fastest reasoning LLM, powered by diffusion by nubg on Tuesday February 24, 2026 @08:22PM (#47146083)
Interesting perspective. Perhaps also the user would adopt his queries knowing he can only to small (but very fast) steps. I wonder who would win!
Re:Mercury 2: The fastest reasoning LLM, powered by diffusion by josephg on Tuesday February 24, 2026 @09:01PM (#47146357)
Yeah I agree with this. We might be able to benchmark it soon (if we can’t already) but asking different agentic code models to produce some relatively simple pieces of software. Fast models can iterate faster. Big models will write better code on the first attempt, and need less loop debugging. Who will win?

At the moment I’m loving opus 4.6 but I have no idea if its extra intelligence makes it worth using over sonnet. Some data would be great!

Re:Mercury 2: The fastest reasoning LLM, powered by diffusion by estsauver on Tuesday February 24, 2026 @10:46PM (#47147060)
For what it's worth, most people already are doing this! Some of the subagents in Claude Code (Explore, I think even compaction) default to Haiku and then you have to manually overwrite it with an env variable if you want to change it.

Imagine the quality of life upgrade of getting compaction down to a few second blip, or the "Explore" going 20 times faster! As these models get better, it will be super exciting!

Re:Mercury 2: The fastest reasoning LLM, powered by diffusion by embedding-shape on Wednesday February 25, 2026 @04:44AM (#47149450)
> Imagine the quality of life upgrade of getting compaction down to a few second blip, or the "Explore" going 20 times faster! As these models get better, it will be super exciting!

I'm awaiting the day the small and fast models come anywhere close to acceptable quality, as of today, neither GPT5.3-codex-spark nor Haiku are very suitable for either compaction or similar tasks, as they'll miss so much considering they're quite a lot dumber.

Personally I do it the other way, the compaction done by the biggest model I can run, the planning as well, but then actually following the step-by-step "implement it" is done by a small model. It seemed to me like letting a smaller model do the compaction or writing overviews just makes things worse, even if they get a lot faster.

Re:Mercury 2: The fastest reasoning LLM, powered by diffusion by jmvldz on Friday February 27, 2026 @03:09PM (#47184907)
The explore step with Codex-5.3-Spark and Opus 4.6 Fast both feel incredible.
Re:Mercury 2: The fastest reasoning LLM, powered by diffusion by bigbuppo on Tuesday February 24, 2026 @09:13PM (#47146440)
Maybe make that intelligence per token per relative unit of hardware per watt. If you're burning 30 tons of coal to be 0.0000000001% better than the 5 tons of coal option because you're throwing more hardware at it, well, it's not much of a real improvement.
Re:Mercury 2: The fastest reasoning LLM, powered by diffusion by estsauver on Tuesday February 24, 2026 @10:44PM (#47147046)
I think the fast inference options have historically been only marginally more expensive then their slow cousins. There's a whole set of research about optimal efficiency, speed, and intelligence pareto curves. If you can deliver even an outdated low intelligence/old model at high efficiency, everyone will be interested. If you can deliver a model very fast, everyone will be interested. (If you can deliver a very smart model, everyone is obviously the most interested, but that's the free space.)

But to be clear, 1000 tokens/second is WAY better. Anthropic's Haiku serves at ~50 tokens per second.

Re:Mercury 2: The fastest reasoning LLM, powered by diffusion by volodia on Tuesday February 24, 2026 @09:21PM (#47146492)
We agree! In fact, there is an emerging class of models aimed at fast agentic iteration (think of Composer, the Flash versions of proprietary and open models). We position Mercury 2 as a strong model in this category.
Re:Mercury 2: The fastest reasoning LLM, powered by diffusion by estsauver on Tuesday February 24, 2026 @10:39PM (#47147011)
Do you guys all think you'll be able to convert open source models to diffusion models relatively cheaply ala the d1 // LLaDA series of papers? If so, that seems like an extremely powerful story where you get to retool the much, much larger capex of open models into high performance diffusion models.

(I can also see a world where it just doesn't make sense to share most of the layers/infra and you diverge, but curious how you all see the approach.)

Re:Mercury 2: The fastest reasoning LLM, powered by diffusion by estsauver on Tuesday February 24, 2026 @09:36PM (#47146586)
I think there's clearly a "Speed is a quality of it's own" axis. When you use Cereberas (or Groq) to develop an API, the turn around speed of iterating on jobs is so much faster (and cheaper!) then using frontier high intelligence labs, it's almost a different product.

Also, I put together a little research paper recently--I think there's probably an underexplored option of "Use frontier AR model for a little bit of planning then switch to diffusion for generating the rest." You can get really good improvements with diffusion models! https://estsauver.com/think-first-diffuse-fast.pdf

Re:Mercury 2: The fastest reasoning LLM, powered by diffusion by refulgentis on Tuesday February 24, 2026 @09:41PM (#47146621)
I'm very worried for both.

Cerebras requires a $3K/year membership to use APIs.

Groq's been dead for about 6 months, even pre-acquisition.

I hope Inception is going well, it's the only real democratic target at this. Gemini 2.5 Flash Lite was promising but it never really went anywhere, even by the standards of a Google preview

Re:Mercury 2: The fastest reasoning LLM, powered by diffusion by behnamoh on Wednesday February 25, 2026 @12:06AM (#47147553)
Once again, it's a tech that Google created but never turned into a product. AFAIK in their demo last year, Google showed a special version of Gemini that used diffusion. They were so excited about it (on the stage) and I thought that's what they'd use in Google search and Gmail.
Re:Mercury 2: The fastest reasoning LLM, powered by diffusion by refulgentis on Thursday February 26, 2026 @05:33PM (#47172982)
Google did not create it; it is correct there was a Gemini that used diffusion, you could apply for access (not via API). It was okay
Re:Mercury 2: The fastest reasoning LLM, powered by diffusion by ainch on Tuesday February 24, 2026 @10:09PM (#47146802)
I don't think it's a good comparison given Inception work on software and Cerebras/Groq work on hardware. If Inception demonstrate that diffusion LLMs work well at scale (at a reasonable price) then we can probably expect all the other frontier labs to copy them quickly, similarly to OpenAI's reasoning models.
Re:Mercury 2: The fastest reasoning LLM, powered by diffusion by refulgentis on Tuesday February 24, 2026 @10:12PM (#47146824)
Definitely depends on what you're buying, maybe some of the audience here was buying Groq and Cerebras chips? I don't think they sold them but can't say for sure.

If you're a poor schmoke like me, you'd be thinking of them as API vendors of ~1000 token/s LLMs.

Especially because Inception v1's been out for a while and we haven't seen a follow-the-leader effect.

Coincidentally, that's one of my biggest questions: why not?

Re:Mercury 2: The fastest reasoning LLM, powered by diffusion by nl on Tuesday February 24, 2026 @09:59PM (#47146745)
Taalas is interesting. 16,000 TPS for Llama on a chip.

https://taalas.com/

Re:Mercury 2: The fastest reasoning LLM, powered by diffusion by micw on Wednesday February 25, 2026 @12:35AM (#47147727)
On a very old model, it's more like 16.000 garbage words/s
Re:Mercury 2: The fastest reasoning LLM, powered by diffusion by nl on Wednesday February 25, 2026 @12:58AM (#47147874)
Llama 3.1 8B is pretty useful for some thing. I use it to generate SQL pretty reliably for example.

They are doing an updated model in a month or so anyway, then a frontier level one "by summer".

Re:Mercury 2: The fastest reasoning LLM, powered by diffusion by numeri on Wednesday February 25, 2026 @10:24AM (#47152765)
but Taalas had to quantize Llama 3.1 8B to death to get it to fit. It can't produce coherent non-English text at all.
Re:Mercury 2: The fastest reasoning LLM, powered by diffusion by patapong on Wednesday February 25, 2026 @06:01AM (#47149970)
I do wonder if there are tasks where 16k garbage words/s are more useful than 200 good words per second. Does anyone have any ideas? Data extraction perhaps?
Re:Mercury 2: The fastest reasoning LLM, powered by diffusion by pnocera on Wednesday February 25, 2026 @02:45PM (#47156782)
A politician communication agent maybe...
Re:Mercury 2: The fastest reasoning LLM, powered by diffusion by DeathArrow on Wednesday February 25, 2026 @01:07AM (#47147929)
I wonder how many token per seconds can they get if they put Mercury 2 on a chip.
Re:Mercury 2: The fastest reasoning LLM, powered by diffusion by replete on Wednesday February 25, 2026 @02:29AM (#47148489)
Its exciting to see, but look at the die size for only an 8b model
Re:Mercury 2: The fastest reasoning LLM, powered by diffusion by Nihilartikel on Wednesday February 25, 2026 @07:41AM (#47150763)
Neat! I had been wondering if anyone was trying to implement a model in silico. We're getting closer to having chatty talking toasters every day now!
Re:Mercury 2: The fastest reasoning LLM, powered by diffusion by empath75 on Wednesday February 25, 2026 @08:14AM (#47151063)
Re:Mercury 2: The fastest reasoning LLM, powered by diffusion by Nihilartikel on Saturday February 28, 2026 @03:19PM (#47199775)
Re:Mercury 2: The fastest reasoning LLM, powered by diffusion by freeqaz on Tuesday February 24, 2026 @09:51PM (#47146683)
You can call Cerebras APIs via OpenRouter if you specify them as the provider in your request fyi. It's a bit pricier but it exists!
Re:Mercury 2: The fastest reasoning LLM, powered by diffusion by andai on Tuesday February 24, 2026 @10:57PM (#47147139)
I used their API normally (pay per token) a few weeks ago. Their Coding Plan appears to be permanently sold out though.
Re:Mercury 2: The fastest reasoning LLM, powered by diffusion by estsauver on Tuesday February 24, 2026 @10:32PM (#47146960)
I am currently using their APIs on a paygo plan, I think it might just be a capacity issue for new sign ups.
Re:Mercury 2: The fastest reasoning LLM, powered by diffusion by 7thpower on Tuesday February 24, 2026 @10:03PM (#47146769)
What do you mean by Grow is dead since about 6 months ago? Not refuting your point, but I’m curious.
Re:Mercury 2: The fastest reasoning LLM, powered by diffusion by refulgentis on Tuesday February 24, 2026 @10:08PM (#47146799)
No new model since GPT-OSS 120B, er maybe Kimi K2 not-thinking? Basically there were a couple models it normally obviously support, and it didn't.

Something about that Nvidia sale smelled funny to me because the # was yuge, yet, the software side shut down decently before the acquisition.

But that's 100% speculation, wouldn't be shocked if it was:

"We were never looking to become profitable just on API users, but we had to have it to stay visible. So, yeah, once it was clear an Nvidia sale was going through, we stopped working 16 hours a day, and now we're waiting to see what Nvidia wants to do with the API"

Re:Mercury 2: The fastest reasoning LLM, powered by diffusion by vessenes on Wednesday February 25, 2026 @10:31AM (#47152890)
The groq purchase was designed to not trigger federal oversight of mergers, so you buy out the ‘interesting’ part, leave a skeleton team and a line of business you don’t care about -> no CFIUS, no mandatory FTC reporting -> smoother process.
Re:Mercury 2: The fastest reasoning LLM, powered by diffusion by Leynos on Wednesday February 25, 2026 @04:06AM (#47149157)
Cerebras are on OpenRouter.
Re:Mercury 2: The fastest reasoning LLM, powered by diffusion by dmichulke on Wednesday February 25, 2026 @12:02AM (#47147531)
Useful for evaluating people as well
Re:Mercury 2: The fastest reasoning LLM, powered by diffusion by jdthedisciple on Wednesday February 25, 2026 @01:14AM (#47147962)
Interesting suggestion.

Maybe we could use some sort of entropy-based metric as a proxy for that?

Re:Mercury 2: The fastest reasoning LLM, powered by diffusion by jakubtomanik on Wednesday February 25, 2026 @05:18AM (#47149709)
Intelligence per second is a great metric. I never could fully articulate why I like Gemini 3 Flash but this is exactly why. It’s smart enough and unbelievably fast. Thanks for sharing this
Re:Mercury 2: The fastest reasoning LLM, powered by diffusion by irishcoffee on Wednesday February 25, 2026 @08:57AM (#47151515)
I really thought this was sarcasm. Intelligence per token? Intelligence at all, in a token? We don’t even agree on how to measure _human_ intelligence! I just can’t. Artificially intelligent indeed. Probably the perfect term for it, you know in lieu of authentic intelligence.

picard_facepalm.jpg