No. I know for a fact that the take you've stated is incorrect, because I work for a startup that makes 8-digit profit annually on inference, with zero outside investment. I don't know why you've latched on to my comment to put this take into my mouth, because I never said anything resembling it. It can be true that a technology has some use while also being true that there is a bubble around people believing things about the technology that aren't true. The internet was one such example: the dotcom bust did not indicate that internet services were not financially viable, but that there were highly irrational allocations of investment that resulted in a lot of people losing a lot of money.

There is a fundamental difference from being able to profit on running some number of GPUs, and being able to profit from running 1 million GPUs. We do not have a business case for 1 million GPUs. Our startup's premise is that there are economically viable niches to use LLMs as tools, which is in stark contrast to OpenAI and Anthropic who are at least publicly selling the idea that LLMs are a form of genuine intelligence that will replace all human labor if you just give them another $10 billion dollars, and then another $10 billion dollars, and then another $10 billion dollars, .....

You may not have a business case for 1 million GPUs but you are not the only business. You clearly have a decent business case for some level of inference as do many others.

The gap between how many GPUs people have a profitable business case for and how many are actually out there is, however, a problem. Per their revealed financials for the pending IPO, Anthropic is deeply, deeply in the hole relative to their spending commitments, which are nearing $1 trillion over the next 5 years. OpenAI's do not appear to be much better from the more limited information we have about them.

The bubble popping will actually make business cases for GPUs much more common to come by, when businesses can acquire them for closer to their cost of production rather than at the insane markups Nvidia currently lists them for. But that won't do Nvidia much good, because Nvidia's valuation is predicated on being able to "sell" as many GPUs as they can produce at a 10x markup over the cost of production - even if much of what is being "sold" is really being given away for equity in businesses that are not financially viable.

> The gap between how many GPUs people have a profitable business case for and how many are actually out there is, however, a problem.

Is this a hypothetical gap or do you have actual numbers to back this up?

The rest of that paragraph continued on to mention two of the biggest users not actually having a profitable business case for the amount of GPUs they have.

That is a huge logical leap you are making then and it isn't supported by the evidence you gave. Are you suggesting that that two companies having more GPUs than they can use is proof GPUs aren't useful at their current price point?

There is a limit to how many GPUs can be profitable at this price point. Suppose you're running a SaaS. You might be able to charge $20/mo and get a sufficient number of customers to cover X amount of GPUs. This does not mean the market can support X * 100 GPUs profitably. THere is a limit to what the market can bear. THere are not infinite customers, in any niche, who will pay $20/mo for an LLM-based SaaS. At some point you will saturate your market of potential customers and purchasing more GPUs will not benefit you. And this even applies cross-SaaS. There is a finite pool of any amount of customers who will pay $20/mo for LLM services. If someone is subscribing to a competitor, they are less likely to pay $20/mo to you as well. All of this to say, it is certainly possible to oversupply GPUs that have a 10x markup. If the price is closer to production cost, then maybe they're cheap enough that you can offer a $10/mo plan profitably, and reach a much wider pool of customers and benefit from more GPUs. But there is a finite number of GPUs at which point the market for a sustainable business cases is satiated.

(Consumer sub prices were used for illustrative purposes but this obviously applies to token and enterprise spending as well. Although they're being a little loose due to FOMO, enterprises will not sign blank checks forever if the promised "replace all human labor" does not materialise)

Also, it goes without saying that a GPU that can not even be powered on for lack of power capacity is not being used profitably, at any price point. It could not be more apparent there is an oversupply relative to its current market price, which is a direct result of Nvidia giving away boatloads of GPUs for stakes in unprofitable companies.

OpenAI is in much worse shape because at least Anthropic has the growth to try to justify the spend. I can't see how the economics work but it is at least in some kind of margin of error.

OpenAI just seems like a black hole for money.

I think what you are describing is the sociological/ecological function of a bubble. We pull an unreasonable amount of demand forward, someone else pays for it and when the bubble implodes the good companies that survive the stress test get a free lunch.

Best of luck at the start up. That all sounds right to me. Real intelligence is specialized and ecological. The grand irony to me is that I would love to be an organism that can walk the way I do but also be able to fly like a bird and swim like a shark.

The problem is that organism would be massively too expensive to survive.

OpenAI's are worse and the market thinks so; they are having trouble raising the money they want, the way they want.

Agree with the rest of this though. This whole idea that Nvidia is likely to get much of a normal return from their vendor financing strikes me as tenuous, and what they really are getting is the creation of the perception of scarcity.

One day soon, the notion that GPU scarcity is a proxy for AI demand will finally lose its currency and I think that will be the start of the reckoning.

It’s not the digital world where you just duplicate an object in memory. You need to find the power somewhere (new contract, generator installation), route it, add management tools (and maybe technicians), and then plug the new gpus. It’s all physical work, which also does cost money, take time, and maybe constrained.

ah this is the category error of the whole bubble.

We are so use to the digital world and the efficiency of electronic markets that it is hard to think of physical constraints in the real world.

> Do you truly believe that the cost that can be charged for inference is below the cost of electricity?

That isn't the issue - it's that building brand new datacentres + the power generation to run them takes a massive amount of time/permitting/money, and by the time those datacentres actually come online, you can be sure that the price pressure on inference will be sharply negative.