Re:I struggle to believe that AI R&D is ingenious
by Kotlopou on Friday November 06, 2026 @07:59AM
(#50391805)
Aside from everything else, you can't just add together the hours from an agent swarm, because swarms become less cost-effective at scale. There's a nice study of that here: https://www.tobyord.com/writing/swarm-scaling
With the most optimistic scaling law from that source, a swarm of 10,000 only gives you a ~500x improvement in performance, and with the most pessimistic only a ~90x improvement. The point of a swarm is to get the same result faster, but more expensively (which OpenAI wanted because of the race dynamics around Navier-Stokes).
This alone knocks 1-2 OOMs off your supposed improvement, and the rest may easily be the different inherent difficulty of the constructions, which is all over the place -- in the big dump, one result (Quasi-RH) took more effort than the rest, and already with Astra people are finding crazy results. Right now, it's easier to judge problem difficulty in terms of the AI time needed to solve them (And whether they can be solved! Most still can't!) than to judge AIs based on which problems they can solve.
With the most optimistic scaling law from that source, a swarm of 10,000 only gives you a ~500x improvement in performance, and with the most pessimistic only a ~90x improvement. The point of a swarm is to get the same result faster, but more expensively (which OpenAI wanted because of the race dynamics around Navier-Stokes).
This alone knocks 1-2 OOMs off your supposed improvement, and the rest may easily be the different inherent difficulty of the constructions, which is all over the place -- in the big dump, one result (Quasi-RH) took more effort than the rest, and already with Astra people are finding crazy results. Right now, it's easier to judge problem difficulty in terms of the AI time needed to solve them (And whether they can be solved! Most still can't!) than to judge AIs based on which problems they can solve.