News for hackers. Stuff that matters.
Re:LFM2.5-24B-A2B: Built for Fast On-Device Agents by nico on Wednesday October 14, 2026 @08:47PM (#50097719)
> with 184 tokens/s on an Apple M5 Max

That seems pretty decent. How many tokens/s do you need for a coding agent to feel like Claude or codex?

Re:LFM2.5-24B-A2B: Built for Fast On-Device Agents by x312 on Wednesday October 14, 2026 @08:52PM (#50097767)
Claude code is 60-80 tps, codex varies by model but mostly 40-50 tps.
Re:LFM2.5-24B-A2B: Built for Fast On-Device Agents by serf on Wednesday October 14, 2026 @08:50PM (#50097753)
>How many tokens/s do you need for a coding agent to feel like Claude or codex?

it's kind of a loaded question, this model gets a lot of tok/s but reasoning is forced on; the time-to-answer is really a combination of things like "how much does this thing want to over-reason" and "how much does this thing like tool use?" .

Re:LFM2.5-24B-A2B: Built for Fast On-Device Agents by nico on Wednesday October 14, 2026 @08:58PM (#50097808)
You are right, they are not exactly the same thing, just wanted to have a reference point

When running locally usually I have to trade off speed for a better model. I can run decently big models at about 2tk/s, or a pretty dumb one at 40-100tk/s

Re:LFM2.5-24B-A2B: Built for Fast On-Device Agents by serf on Wednesday October 14, 2026 @09:08PM (#50097887)
fwiw this thing puts out results very fast even on max reasoning.

the results i've gotten so far have been a little braindead though. still playing around. bad luck with traversing filesystems remotely so far.

it seems to want to make concluding decisions very fast regardless of reasoning levels.

Re:LFM2.5-24B-A2B: Built for Fast On-Device Agents by davidsainez on Wednesday October 14, 2026 @09:14PM (#50097912)
AFAIU prefill speed makes a bigger difference in agentic coding workflows as the harness likely spends more time reading than creating new code. Qwen in particular spends a lot of time reasoning so it does make a larger impact there
Re:LFM2.5-24B-A2B: Built for Fast On-Device Agents by nightbun on Wednesday October 14, 2026 @08:49PM (#50097740)
I’d say only 65
Re:LFM2.5-24B-A2B: Built for Fast On-Device Agents by isomorphic on Thursday October 15, 2026 @04:01AM (#50100689)
Another data point: I got 94.74 tokens/s on an M4 Pro Mac mini 64GB with the simonw prompt "Generate an SVG of a pelican riding a bicycle". TTFT was 0.37s. The model thought for 40.75s before answering.

(The resulting pelican was underwhelming.)

Re:LFM2.5-24B-A2B: Built for Fast On-Device Agents by Tade0 on Thursday October 15, 2026 @04:20AM (#50100803)
200-250 is the rate I've seen my context window fill up during longer tasks with Claude. The 40-80 figure is what Google's AI mode says, but I don't know where it got that from.

I've been messing around low computing power hardware to run models locally and to me 100 is already plenty fast. You most likely won't notice a huge difference between 184 and 250, unless you're running some serious multi-age agent circus.

Re:LFM2.5-24B-A2B: Built for Fast On-Device Agents by kerenskiy on Thursday October 15, 2026 @08:57AM (#50102986)
The context window fills up also/mostly from prefil which is supposed to be 20+ times faster or so. I think 40-80 tps for decode/output is believable for claude models