News for hackers. Stuff that matters.

A comment on Edge0-145B: Frontier AI Capabilities on Your Own Computer.

Re:Edge0-145B: Frontier AI Capabilities on Your Own Computer by bob1029 on Friday November 13, 2026 @02:50AM (#50479823)
Frontier AI capabilities also include practical token rates. My current workload would require around 300 tokens/s continuously looking at the last 30 days of token spend. More realistic would be 1500-2000/s for human in the loop activities. I can't babysit the thing 24/7.

How far are we from this kind of throughput with local models?

[ Parent | on HN ]
Re:Edge0-145B: Frontier AI Capabilities on Your Own Computer by iCarrot on Friday November 13, 2026 @04:01AM (#50480416)
Isn't this pretty much backward? If the thing spit out codes so fast, you'd have to babysit it. Slower throughput let you do something else while you wait.

Also throughput pretty much depends on compute capabilities and active weight. So if someone build a A0.1B then maybe you can look at 1500t/s of pure uselessness.

[ Parent | on HN ]
Re:Edge0-145B: Frontier AI Capabilities on Your Own Computer by bob1029 on Friday November 13, 2026 @09:20AM (#50483878)
What it is spitting out is mostly not code anymore. I think much of the bubble narrative around HN is borne out of this perception that everything anyone would ever do with AI is related to code.
[ Parent | on HN ]