Part of me wonders (maybe naively) if with falling cost of AI inference and improvement of models means there is an oversupply of inference infrastructure since we may reach a point where no one needs more and everything you could want can be done by the machine and adding more compute for most users is not any kind of benefit (like it was before LLMs there was an obvious ceiling where the majority of people did not need more compute.). I know Jevons paradox is a thing and I could see it going the other way as well but the first possibility seems to be ignored and I am not sure we can. Eventually demand for training might dry up as well if are models become largely very good and only a bit of fine tuning is required for most applications.

Some years ago I came across this site: https://sites.google.com/site/1024cores

The author is a developer working on software that scales to 1,024 cores and greater -- which is within spitting distance now thanks to 256-core AMD EPYC processors. If you count hyperthreads as a "virtual core" then a dual-socket system gets you there!

That blog made me realise that almost all software can't utilise 1,024 cores, not even close! Microsoft SQL Server, as a random example, tops out at 64 cores for any single query.

Similarly, people couldn't utilise a machine that big, not with pre-AI software, except for certain embarrassingly parallel codes such as simulations and 3D graphics rendering.

Now... NOW... there's no upper limit. None. One person(!) can cheerfully utilise millions of cores, billions even, and there's no end in sight.

If you think one conversation thread in "ChatGPT" is all we need, you're not seeing the big picture of what's possible here.

Those math proofs OpenAI produced were created with tens of thousands of agents running in parallel, orchestrated by thousands more agents, and overseen by hundreds of agents, all triggered by someone typing as little as a sentence or two to start that avalanche of compute!

With that kind of AI power at your fingertips, huge, unthinkable projects become a fun exercise for the weekend:

- Translate the entirety of Wikipedia to every human language.

- Rewrite the entire Linux kernel to Rust in one go

- Rewrite all open source C code to Rust.

- Formalise every maths paper ever written.

Etc, etc, ... without end, without bound.

I am arguing that their is an quantity intercept (The demand for compute if it cost nothing would be a finite number and may go down rather then up with advancements in cheaper inference and accuracy).

Sure you can do all these things you mentioned but eventually even if it was free they would be done and doing some of these things your self would be annoying and a waist of time. Once some has formalized a proof why would some do it again and why would some one rewrite a program again, it will still take time and human effort over just using it. If you just spend 1 minute with each agent of your say 10000

I really don't think most people want to be running tens of thousands of agents and would not find a need for that many agents. if you it takes you just 1 minute of looking at output of your agents that would take you 1 year and 52 days. I think you have to admit there is a limit. Though I could find busy work for my agents, you could do the same with computing before hand such searching for primes, there is a limited thing that people want out of their machines is my point and people who would run such things would be a minority if they exist at all.

That’s not how it works. Agents coordinate agents, validate results, fan out work, coalesce work, and hand off to other agents. It’s agents all the way down.

Your other point stands tho - there might be a point where there is not much left to do. Most problems solved, and remaining require more agent hours than there is energy in the universe. Or not. Who knows.

Multi-agent cooperation effectinveness scales logarithmically with cost, except with tasks that are embarrassingly parallel.

I share your enthusiasm, but only directionally. :)

I don't see how it helps to think so narrowly about the computation needed solely for training when we are clearly headed to a future where tons of realtime applications (think robots) need to do insane amounts of processing ASAP while doing time-sensitive tasks.

IMO its copium to think that chips which have 10x more demand than supply are suddenly going to be easy to get as "AI is eating the world" like software did over the past decades.

You are definitely right, mrgordon - this isn't an infinite scaling, "one extra copy cost essentially nothing," software issue. Video cards are physical. You can't just sell more copies with almost zero marginal cost like software. Each video card is a physical good that is subjected to all those physical laws.

I was not trying to deny the corporeality of GPU's (or cpu and memory), only that there might be an oversupply given the rate of buy up of these things and the amount of start ups who have brought these things and will fail and liquidate or liquidate hardware as they pivot or go into just offering inference.

Secondly that the frontier model providers will not necessarily want to buy these up as the aforementioned lower demand for training (or costly and memory intensive training) and cost of inference goes down (and possibly requests as the models get better or at least plateau).

Further TPU's and if model weights do eventually stabilize then ASIC's might make more sense.

And lastly as models improve and make less mistakes the requirement trying things in parallel traditional compute may also go down. I think their is a quantity intercept (The demand for compute if it cost nothing would be a finite number and may go down rather then up with advancements in cheaper inference and accuracy) for compute both AI inference and traditional compute.

I think there is a question of how much of that demand is real. If GPUs are sitting unused in boxes and/or if NVDA has been artificially juicing demand either through basically buying them from themselves or selling them to China through an intermediary then the 10x demand comes into question.

That leaves the real buyers who are also starting to run low on capital.

Finally, demand doesn't need to go to zero for there to be a price correction. Just dipping from insane demand to elevated levels would lead to a significant price drop.

real time visual data processing for these applications seems to me to not be that expensive and the thinking part will be done as a swarm. There is some real quantity intercept for the demand of compute that does change over time but one can imagine it going down with better inference not to mention ASIC's.

I think there are two sides that want a particular outcome one want gpu's to get cheaper (like I admit I do), others want there investment to work out (both might want to huff copium at some point, who is huffing here is not easy to tell).