Look at DeepSeek! They have done full system optimization on their model, architecting it so that it would be cheap to serve. I am proud to call DeepSeek a world class tech engineering firm in a way that 99% of the tech world is not. They are a 1% company.

For the time being this thesis hasn’t worked out though. H100’s are only going up in price and becoming more scarce [1]. At what point would you update if now going on 3 years into H100 lifecycle this doesn’t seem close to happen.

But they are becoming more scarce while staying on Nvidia’s shelves buy-and-hold, or being leased back in some cases. They are not being installed by customers at anything like the rate the sales figures suggest. So yeah, these H100s will last a long time in the packaging.

Meanwhile, Meta is selling their spare AI data centre capacity.

Isn’t that what SpaceX is doing too? Selling spare compute while they work on their rocket side hustle.

Where are all these unused H100s sitting and where are the people who have been fired for buying them?

They are in Nvidia warehouses. Nvidia is selling H100s and other hardware in advance to firms who do not have data centres to put them in.

Nobody is getting fired for thinking they need to buy scarce H100s in case they aren't able to get them in the future. Nobody is even getting fired for keeping their companies out of Jensen Jail — the fear that if they don't buy the whole server stack from Nvidia they might not get future GPU allocations.

There really are analysts who think Nvidia is channel-stuffing.

If the market deflates there will be a glut of very cheap compute, both in terms of data centres (though many of them will be able to offer managed model training services instead, I suspect) and in terms of unused hardware being made available for sale.

Nvidia won't lose out; everyone else is going to.

The market for compute or the stock market? People seem to have got lost conflating these two things.

Supply and demand bring what it is means that (alleged) cheap compute won’t stay cheap or go unused for very long.

Both the market for compute and the stock market valuations of AI firms (and theoretical valuations) are overcooked.

The problem is that "scarcity of GPUs and RAM" is being used as a proxy for "demand for AI".

It's a broken proxy; the scarcity of H100s is artificial.

Here's interesting question I haven't seen covered. The promises to buy hardware (not just NVIDIA but networking, servers themselves, etc etc) are NOT on the income statements of the buyers...

Are the contracts on the income statements of the sellers? If Anthropic promised to buy $200B of GPUs from NVIDIA, is it recorded on NVIDIA's income statement (other than a footnote "we have $200B orders lined up")?

'Cause if it is, that'd be $200B GDP increase _literally_ conjured out of thin air.

I don't agree with him on everything but one of the questions Ed Zitron has raised is — have all the GPUs Nvidia has sold actually been manufactured and warehoused?

If they haven't been manufactured, things are potentially quite sketchy.

Jevons' Paradox. If it becomes more efficient, cost goes down, demand goes up, use rises, revenue rises.

no, it's 10y. you just repurpose.

also, power and cooling agreements are usually 10y too.

> The current-gen hardware is expected to have only 2-3 years of useful life.

Expected by whom and why?

Google hasn't retired any of its now ten-year-old TPUs, or any of the subsequent generations. Maybe TPUs are special or Google doesn't understand compute economics but either argument requires data.

If your argument is depreciation, you're confusing a tax concept (depreciation) with useful life. Yes, they're supposedly related, but in practice, the relationship is rather loose. (Residential housing depreciates in 28 years yet typically lasts 2-3x as long and 4x is not rare.) Google's V1 TPUs

People are buying v100s on ebay to run LLMs and Chinese manufacturers make custom PCIE boards for them. Older cards might not be useful for training but they're perfectly capable of doing inference work.