I think stall would mean not being willing to ramp up the spending at each generation, unless there is corresponding return on investment within grasp.

Say in a couple of years Microsoft and Amazon have each just bet another $10B on their respective horses, and resulting model performance gains are leveling off towards some limit (with no secret backroom glimpse of a breakthrough on the horizon). Do they keep on pumping in another $10B a year to eke out those diminishing returns? Perhaps, but that would seem best case. It would be hard to justify spending $100B on next generation just hoping for a miracle to happen.

So, if this is the way it pans out, with nobody willing to fund continued model scaling due to diminishing returns in terms of performance gains, then any further AGI advance would have to wait for changes in approach that are less expensive to pursue.

If the spend is X billion [inflation-adjusted] dollars per year on AI, we still get exponential returns on compute. That is not a stall in compute, obvs. So the question is, will that exponential gain in compute will yield exponential gains in AI capability? I suppose it depends on how you measure capability. I don't know how to measure that. But even if it only yields linear gains in AI capability, that is still not a stall, by any definition.

Right now it seems model-size/training-tokens are roughly going up 10x per year, resulting in training costs going up 10x per year. As long as investors are willing to increase spend 10x year over year then models can continue to grow 10x year over year.

The "stall" scenario is where model performance from one year to the next, despite 10x increase in size, stops getting significantly better (i.e. better to extent that investors project increased revenue and worthwhile ROI). Let's say best case scenario here is that investors are willing to keep putting money in, but only at a flat year-over-year level. Without that 10x YoY spend increase, the model designers have lost a 10x factor in ability to increase model size. Perhaps compute prices halve YoY, so they can still get a 2x model size increase, but what will this do if prior year's 10x increase just saw performance leveling off?

This seems like a best case scenario in this "performance levelling off to point of minimal RoI" situation. Why would an investor "throw good money after bad" and spend another $10B after having judged that last years $10B wasn't really worth it?

So, under this logic, it seems that either:

a) scaling/tweaks are all you need, and YoY performance gains continue to be impressive and support 10x model-size increases/spend

b) scaling is not all you need, but some AGI-critical breakthroughs are made BEFORE investors give up on current trajectory

c) scaling is not all you need, the AGI breakthroughs are not made in next few iterations, and the industry enters "stall" mode (i.e. no further progress towards AGI, other than ongoing research to find a new direction)

Maybe the slowdown would actually result in us surviving!

Maybe - definitely safer if it's slowly introduced!