But presumably those gpus are being actually used by those companies? Even if their prices are artificially inflated? Or are these buyers sitting on piles of unused gpus to make demand look good?

I mean even Satya Nadella famously said that Microsoft has plenty of GPUs just nowhere to put them. There are definitely plenty of unused GPUs out there. Jensen is also a big fan of scarcity (said in multiple interviews) so obviously he’d want to keep it up at all costs.

> These GPUs are probably in use now,

A different question might be, will these companies grow over the new few years?

If I give you 10 Million GPU in exchange for 25% of the company and the company goes bust or has negative growth (and thus is forced to sell assets), effectively I just put 10M GPUs on the market with a huge discount.

At a grand enough scale, this undermines the GPU marker and one could argue that Nvidia has been doing this on a massive scale.

> Or are these buyers sitting on piles of unused gpus?

Yes, they are literally doing exactly that. Noting that they aren't truly "buyers" so much as "gift recipients" (in exchange for ownership stake, but the value of their remaining ownership also goes up if these gifts generate sufficient hype for investors to invest more). GPU production is outstripping datacenter capacity.

> But presumably those gpus are being actually used by those companies?

But, even if they were all being used, this wouldn't mean much. The central point is: suppose Nvidia has produced 10 million[1] datacenter-class GPUs during the LLM bubble. Nobody can afford to, or is willing to, buy 10 million datacenter-class GPUs because the economics don't work out. But if they gift 10 million GPUs, and frame it like they sold the GPUs (a framing that is effective -- although this has been an open secret, this is being reported like news by the WSJ, and to the majority of their readers, it is news), it makes Nvidia GPUs appear to be extremely desirable. Even though, logically, an item that you have to give away because it's not economically worth its market price is the exact opposite. The fact that you can use goods that you've effectiely been given for free does not increase the actual economic value of those goods.

[1] (hypothetical number, it's vaguely around this order of magnitude)

Why would they sit on piles of unused GPUs when they could rent them out or use them? Even if they were free that is still not what a business would do with them.

"Why don't they just use their unused GPUs"? - because they literally cannot. GPUs require electricity. They have more GPUs than they can supply electricity to.

https://www.datacenterdynamics.com/en/news/microsoft-has-ai-...

> Microsoft CEO Satya Nadella has said the company has AI GPUs sitting idle because it doesn’t have enough power to install them

https://www.businessinsider.com/amazon-nvidia-ai-gpu-data-ce...

> Some AWS sites had room for more racks but not enough available power to turn them on

> At least one AWS team did not have "a clear forecasted path" for getting the incoming racks into data centers at the time

Do you truly believe that the cost that can be charged for inference is below the cost of electricity?

No. I know for a fact that the take you've stated is incorrect, because I work for a startup that makes 8-digit profit annually on inference, with zero outside investment. I don't know why you've latched on to my comment to put this take into my mouth, because I never said anything resembling it. It can be true that a technology has some use while also being true that there is a bubble around people believing things about the technology that aren't true. The internet was one such example: the dotcom bust did not indicate that internet services were not financially viable, but that there were highly irrational allocations of investment that resulted in a lot of people losing a lot of money.

There is a fundamental difference from being able to profit on running some number of GPUs, and being able to profit from running 1 million GPUs. We do not have a business case for 1 million GPUs. Our startup's premise is that there are economically viable niches to use LLMs as tools, which is in stark contrast to OpenAI and Anthropic who are at least publicly selling the idea that LLMs are a form of genuine intelligence that will replace all human labor if you just give them another $10 billion dollars, and then another $10 billion dollars, and then another $10 billion dollars, .....

You may not have a business case for 1 million GPUs but you are not the only business. You clearly have a decent business case for some level of inference as do many others.

The gap between how many GPUs people have a profitable business case for and how many are actually out there is, however, a problem. Per their revealed financials for the pending IPO, Anthropic is deeply, deeply in the hole relative to their spending commitments, which are nearing $1 trillion over the next 5 years. OpenAI's do not appear to be much better from the more limited information we have about them.

The bubble popping will actually make business cases for GPUs much more common to come by, when businesses can acquire them for closer to their cost of production rather than at the insane markups Nvidia currently lists them for. But that won't do Nvidia much good, because Nvidia's valuation is predicated on being able to "sell" as many GPUs as they can produce at a 10x markup over the cost of production - even if much of what is being "sold" is really being given away for equity in businesses that are not financially viable.

> The gap between how many GPUs people have a profitable business case for and how many are actually out there is, however, a problem.

Is this a hypothetical gap or do you have actual numbers to back this up?

The rest of that paragraph continued on to mention two of the biggest users not actually having a profitable business case for the amount of GPUs they have.

That is a huge logical leap you are making then and it isn't supported by the evidence you gave. Are you suggesting that that two companies having more GPUs than they can use is proof GPUs aren't useful at their current price point?

There is a limit to how many GPUs can be profitable at this price point. Suppose you're running a SaaS. You might be able to charge $20/mo and get a sufficient number of customers to cover X amount of GPUs. This does not mean the market can support X * 100 GPUs profitably. THere is a limit to what the market can bear. THere are not infinite customers, in any niche, who will pay $20/mo for an LLM-based SaaS. At some point you will saturate your market of potential customers and purchasing more GPUs will not benefit you. And this even applies cross-SaaS. There is a finite pool of any amount of customers who will pay $20/mo for LLM services. If someone is subscribing to a competitor, they are less likely to pay $20/mo to you as well. All of this to say, it is certainly possible to oversupply GPUs that have a 10x markup. If the price is closer to production cost, then maybe they're cheap enough that you can offer a $10/mo plan profitably, and reach a much wider pool of customers and benefit from more GPUs. But there is a finite number of GPUs at which point the market for a sustainable business cases is satiated.

(Consumer sub prices were used for illustrative purposes but this obviously applies to token and enterprise spending as well. Although they're being a little loose due to FOMO, enterprises will not sign blank checks forever if the promised "replace all human labor" does not materialise)

Also, it goes without saying that a GPU that can not even be powered on for lack of power capacity is not being used profitably, at any price point. It could not be more apparent there is an oversupply relative to its current market price, which is a direct result of Nvidia giving away boatloads of GPUs for stakes in unprofitable companies.

OpenAI is in much worse shape because at least Anthropic has the growth to try to justify the spend. I can't see how the economics work but it is at least in some kind of margin of error.

OpenAI just seems like a black hole for money.

I think what you are describing is the sociological/ecological function of a bubble. We pull an unreasonable amount of demand forward, someone else pays for it and when the bubble implodes the good companies that survive the stress test get a free lunch.

Best of luck at the start up. That all sounds right to me. Real intelligence is specialized and ecological. The grand irony to me is that I would love to be an organism that can walk the way I do but also be able to fly like a bird and swim like a shark.

The problem is that organism would be massively too expensive to survive.

OpenAI's are worse and the market thinks so; they are having trouble raising the money they want, the way they want.

Agree with the rest of this though. This whole idea that Nvidia is likely to get much of a normal return from their vendor financing strikes me as tenuous, and what they really are getting is the creation of the perception of scarcity.

One day soon, the notion that GPU scarcity is a proxy for AI demand will finally lose its currency and I think that will be the start of the reckoning.

It’s not the digital world where you just duplicate an object in memory. You need to find the power somewhere (new contract, generator installation), route it, add management tools (and maybe technicians), and then plug the new gpus. It’s all physical work, which also does cost money, take time, and maybe constrained.

ah this is the category error of the whole bubble.

We are so use to the digital world and the efficiency of electronic markets that it is hard to think of physical constraints in the real world.

> Do you truly believe that the cost that can be charged for inference is below the cost of electricity?

That isn't the issue - it's that building brand new datacentres + the power generation to run them takes a massive amount of time/permitting/money, and by the time those datacentres actually come online, you can be sure that the price pressure on inference will be sharply negative.

Nvidia does lease them back from some of these customers. But it's not clear even Nvidia is plugging them all in.

Car companies do that all the time ever since banking licenses have been issued to non-financial institutions.

Nothing new, nobody pays a car cash anymore

I wonder if the economics of the two industries is similar enough to be useful here. Car demand itself seems fairly stable and repossession is a way of recovering value in cases of default. Credit scores of individuals purchasing cars being tied to salary/wages vs. startups without much history. I just wonder if the analogy isn't fraught with issues.