The compute they sold is based on the idea that the amount of money people spend on inference will continue to go up dramatically.

If we’re in a gas shortage, all that matters is who has the gas, true. But the issue is that we’re burning 50% of the gas that’s made every day, mean while the other 50% is being stockpiled based on the idea that everyone is going to drive 10x farther next year.

Is that true? Aren't they going to use those datacenters for training and robotics as well? Not just chatbot inference? GPUs aren't Bitcoin miner ASICs, they can do a wider variety of things.

Gasoline in a car is one analogy, but another one is data on the Internet. When the Internet first broke out of universities, there was dial-up. In the late 70's the 300 baud modem was being sold to consumers. It would take roughly three hours to download a novel of 350 pages. It would take roughly two hours to download a single poor quality 256 kilobyte raw 8bpp image file. Jpeg hadn't been invented yet. Now, that image file will take less than ten milliseconds. Roughly a million times faster.

The things we transferred changed too. Today, we stream multiple 4k video with audio streams up and down, so a regular home connection won't suffice for more than a few users. Wirth's law; what Intel giveth, Microsoft taketh away; also applies.

Sometimes digital trends do tend to 10x, or in the case of ISP speeds, we went from 300 baud to gigabit in ~40 years; 3.3 million times faster.

I have no idea what the future holds, but analogies only go so far.

> Is that true? Aren't they going to use those datacenters for training and robotics as well?

Sure, those things too. I was meaning “inference” as a shorthand for any ai workloads.

Whatever analogy you want to use: gas, data transfer, it doesn’t matter. My point is that all the capacity is being hoarded for future use making it feel like a scarce resource now.

If the demand grows like they say it will, then hoarding was a smart choice and the people who own the GPUs will make lots of money. If the demand doesn’t grow as fast and as much as they say, then hoarding was a bad call and the house of cards collapses and a lot of people will lose a lot of money.

To use your data transfer analog - during the dotcom bubble a ton of fiber was built. But after the bubble popped most of it went unused for years. Some of that “dark fiber” is still unused today, 25+ years later. Afaik it would have been a lot cheaper and more stable for the economy to lay that fiber more incrementally.

Except:

1. GPUs are reportedly increasingly just getting put on shelves because there aren’t data centers to host them, which is creating a setup for a financial mess (rapidly depreciating assets purchased, paid for, but not utilized),

2. And massive chunk of the compute “demand” is coming from circular financing (ie AWS is “sold out” in large part because it gave startups like Anthropic money to buy the thing it’s now telling the market is “sold out” so there’s some shenanigans behind those headlines.)

I do agree though that compute is really the only place to make money in AI in the long run. Models are an increasingly a commodity with little value above the cost of compute to run them. However selling compute will be with razor thin margins, not the fat AWS margins the market got used to in cloud, so AWS and others will have to deal with that uncomfortable reality.

I worry they’re going to destroy the chips they can’t afford to deploy to keep them from being used for local inference

How would chips that, say, Amazon paid for end up in local devices?

I don’t understand the scenario that destroying the chips would guard against. Amazon could just leave these chips in a warehouse indefinitely.

I guess this is functionally destroying them? Doesnt take long for chips to go out of date. If I buy a load of potatoes and let them rot, have I not destroyed them?

> If I buy a load of potatoes and let them rot, have I not destroyed them?

Depends on how they're stored. Computer chips aren't potato chips, they don't rot, properly stored they can last a really long time, a multitude of storage technologies are available for that purpose.

> Doesnt take long for chips to go out of date.

That used to be the case but it's no longer true. Advanced chips are made by 3 companies total and competition is nowhere to be seen. Then, with every generation of chips, their production costs and prices are rising more than the benefits that generation provides i.e. the benefits curve is flattening faster than cost curve.

Theoretically, China can bring the prices down but they are sanctioned, tariffed and have a huge market of their own to satisfy first.

> rapidly depreciating assets purchased

GPU's produced in 2020, with a supposedly "3 year life before depreciating to zero" are now sold at twice their price from 2020.

So did houses in 2007. That’s not a good sign at all.

I guess GPUs are interesting because unlike houses, we've suddenly found a massive amount of new utility for these things. If my house suddenly started producing its own furniture, it might be a bit more valuable. Which isn't to say people haven't found new things to do with their houses - COVID made having a home office a lot more valuable, for example, but nothing quite as dramatic as GPUs suddenly able to produce their own code.

In 2008 we had both a massive under supply of housing in many cities plus crashing prices. These things can very much exist at the same time if, as if happening now with GPUs, the financial engine fueling the ecosystem is out of whack.

> 1. GPUs are reportedly increasingly just getting put on shelves because there aren’t data centers to host them, which is creating a setup for a financial mess (rapidly depreciating assets purchased, paid for, but not utilized),

Unfortunately, "paid for, but not utilized" doesn't result in rapid depreciation in the case of coordinated market scalping. In that case, dark (unused) hardware is kept on storage to actually cause higher prices until them chips can be sold for profit, in the form of services, after eventual deployment.

I suspect, digging a bit deeper would find some shady financial practices that may lead to a taxpayer funded bailout via the insurance loophole.