But as a fraction of inference revenue, there is good reason to estimate that the direct electricity costs are perhaps 1-5% of inference revenue.

That’s a lot, but hypothetically if electricity prices double, it’s more likely to lead to angry residential consumers than a doubling of inference token prices.

Lawrence Berkeley National Lab, November 2026: https://eta-publications.lbl.gov/sites/default/files/2026-11...

There are substantial economies of scale on both power generation (grid-scale) and inference power consumption (batching and utilization).

But that’s a feedback loop that could lead to more supply being built. Or it could lead to regulations that put enormous surcharges on data center power consumers, which effectively shield consumers directly from the lack of supply.

This would be the largest biomass plant in the US. 5x larger than the next largest plant.

A 500MW plant would require almost 50x the amount of available waste material from logging and sawmill operations in the entire state of TN (where the plant is being built).

The developer of the project said that the plant was designed to create demand for low grade pine timber following pulp mill closures in the area and that the plant will burn between 350-500 truckloads of logs per day.

IF prices of GPU/RAM come back to pre bubble costs, we might see the 5 year cycle of replacement that's relatively common in normal datacenters pre-ai bubble.

At current prices ? those GPUs that are there now will be running there in 10 years still.

If they don't burn out after two or three. Annualized failure rates are about 9% according to Meta.

Less than half of the original hardware will be running after ten years even if you assume that rate does not increase over time. Do H100s come with replacement guarantees?

This paper puts the failure rate of A100s closer to 0.3%/yr, H100s under 1%.

Mind you, most of the H100s that have ever been sold are still in their packaging, I suspect. Or at least not plugged in.

> [...] because quite frankly, the biggest issue we are now having is not a compute glut, but it’s power — it’s sort of the ability to get the builds done fast enough close to power. So, if you can’t do that, you may actually have a bunch of chips sitting in inventory that I can’t plug in. In fact, that is my problem today. It’s not a supply issue of chips; it’s actually the fact that I don’t have warm shells to plug into.

https://www.tomshardware.com/tech-industry/artificial-intell...

I do agree that 5 years isn't the hard cutoff I was suggesting but I'm not convinced that 10 years is a reasonable timeframe to consider paying the debt down when Moore's Law is involved. I know it's no longer the 12-18mo doubling it used to be but even if it's 2ish years then over 6 years you get 3 doublings or 8x performance gains. At 10 years it's another 2 doublings or 32x performance gains. It's hard for linear payback curves to deal with exponential performance curves past 2-3 doublings.

I guess this is the big gamble, the idea that if you just spend enough money as fast as you can you can become the gods of AI forever and can just charge whatever you want for your magic beans.

But what is the moat? If in reality these things become commodities and just compete with each other to spit out cheap tokens then there's no way the margins are there.

'Winner take all' doesn't mean convincing others to buy your product. If others have something to buy your product with, you've not taken all, fundamentally.

'Winner take all' means having the most powerful army. The assumption is that once AI crosses a certain threshold then there won't be anything the rest of us can do to stop the machine of whomever gets there first, so they will truly be able to take all. This is why the AI labs sometimes talk about the risk of human destruction.

But I am not sure you are right that it is strong enough to secure all. I search daily, but cannot remember the last time I used Google. And I'm not alone. Google's stock price recently took a nosedive because investors have grown quite worried about the competition. Without an army stopping them, there will always be someone else ready to try to outdo you.

Google shares dipped because AI shares dipped, not because their core business at risk. Google is in a _very_ strong position to capitalize on AI. Better than almost any other Big Tech company.

With AI, it looks to me like competition is much tighter, with no company having fundamentally superior technology or monetization plan. So a winner-take-all scenario is much less likely.

Maybe you mean that you could see the employees of those companies staging a coup to seize the army for themselves? That's certainly possible. They'd be in a good position to do so. Predicting the future is impossible, but if you want to be on the "right side" of that future army, owning its foundation today is still a reasonable bet.

Are you saying that winner-take-all markets are incompatible with capitalism? I'm confused

In reality the people would likely revolt by no longer recognizing private ownership of capital if that were to occur, but technically they don't have to. If they truly believe in capitalism they can continue to uphold the right for that private entity to own all the capital.

For the sake of discussion, if we entertain your thought, that means capitalism also doesn't exist beyond a thought experiment, which questions why it was brought into this discussion? Whether or not a winner-take-all-future is compatible with that thought experiment with no actual grounding really doesn't matter, does it?

Still, if you've ever actually dealt with a monopoly, you must surely understand the difference between that and an actual competitive market right?

Yeah, I know. We see you do that below.

> You should really try working in cloud.

I do. Yet I don't need to negotiate with a police force to protect it, so clearly there isn't a free market. But even if you want to pervert what a free market is by cherrypicking some things you like from it while ignoring everything else, I still don't see how you would ever consider the cloud to be free. It is marred by things like copyright and patents that make free exchange unrealistic.

> Our customers churn all the time because they have meaningful choice, and our margins reflect that

Is this humorous story intended to be an arbitrary tangent or something related to the rest of the discussion? The best interpretation I can come up with is that you are trying to use this story to explain that your cloud activities exist in a market, which is no doubt true, but you haven't given anything to suggest it is within a free market. Even non-free markets need to have choice. You cannot have a market at all, neither free nor non-free, if there is no choice. But there is more to markets than actors being able to choose. People having choice was present in societies even before the idea of a market was invented.

I don't think anyone serious believes this any more. The tech is thoroughly commoditized. There are a lot of plays to be made, like OpenAI cornering memory, or spending heavily on unique data for RL, if you can stop distilling, but there is no proprietary core IP at all.

There is no way to monopolize a technology this big. It would be like monopolizing literacy.

More specifically: when everything you could previously have earned money with is subsumed by AI, the "frontier" of profitable knowledge exists solely with the highest concentration of capital. Because that's where the highest amount of compute and consequently valuable data is.

Already now the idea of the tech being "commoditized" is true only if you exclusively look at the bubble of rich countries. AI-worthy compute is prohibitively expensive otherwise for individuals.

It's almost as if arithmetic were the least important part of mathematics. :-)

Here's the thing: we could bring it back. It had a good 50 year run, it could have another. Lina Khan was working on it, you probably saw the flak. This was on the ballot in 2024 and it will be again in 2028.

The latter doesn't imply the former. In other words it doesn't matter what the reality is, it matters what the excuses for sinking money down the hole are.

> There is no way to monopolize a technology this big. It would be like monopolizing literacy.

Again, I don't know if literacy is monopolized or not, that's a separate and quite long conversation, but whatever the case is, it doesn't mean anything wrt AI because GPUs and RAM are already monopolized and successful market scalping requires monopolization.

The bubble is simply an extraction mechanism, the conditions for it are present now, as are the will and power available to those who can do it and benefit from it. How long it's going to continue is a question with many unknowns but the conditions for it can be maintained for at least another 3 to 7 years.

If, and it is a big "if", someone were to become "the gods of AI", they get to perform a "pivotal act", i.e. take over the world. And if they do it wrong, end the world.

From the public statements, I get the impression that Zuckerberg is saying words solely to give positive vibes and does't actually believe any of that; that Musk might believe all of that but has only given as much cursory thought to it as one might get from reading a Wikipedia plot summary of Colossus: The Forbin Project, all The Culture novels, and an elevator pitch (~30 seconds or so) with Yudkowsky in 2013; others have certainly painted Altman in similar light; which leaves Amodei as a solid "C-" in the sense that while Anthropic still isn't taking this even close to seriously enough, he hasn't overtly said or done things that implicitly or explicitly show he thinks it's just fairy tales to get nerds interested in working for him.

People not only ignore second order effects, they also don't think about what would actually happen when a "winner" really "took all".

While AI is a tremendously useful technology, business types wildly misunderstand it and its consequences. Greed eats brain.

I think in 3-5 years close to 100% of current GPUs/RAM will still be in service. New hardware matters only if gains in efficiency are large enough to put you out of business because the competition gets them. Extreme case of that are bitcoin miners. With AI inference there are way too many factors that influence the profitability. It's not like you can't run new model without the new hardware. It can just run on more of the old one. 3-5 years for GPU are estimates when the main application was gaming with new games with new requirements for running on a single GPU being constantly released.

I wouldn't be surprised if most hyperscalers are betting on cheaper GPUs on their next upgrade through either pricing pressure on NVidia, competition from AMD/Broadcom or in-house chips.

https://epoch.ai/data-insights/ai-datacenter-cost-breakdown

https://www.csis.org/analysis/impact-tariffs-ai-data-center-...

The first one says that networking infrastructure costs 13% and the other says 1.5%. Not a great start.

Let's imagine that a traditional full rack can host 80ish H100s and that each one costs $25k. That's 2 million right there. There's probably only 10x cases in that rack so that's 10x dual core motherboards, RAM, network, etc. Call them 20k each. Maybe you need a 32 port 100gig fiber switch, that's 8k. Let's say it's 2.3 million per rack and 2 million of it is just the GPUs. We're at 85% right there.

I suspect that it would take more than just another 100k per rack to finish out a data center so my "80% of the cost is in the GPUs" is definitely wrong. But I didn't see anything in these articles that indicated to me that 50% of the cost of a datacenter is anything really durable like buildings or electrical infrastructure or cooling. There's still an enormous amount of cost in stuff whose performance follows more of an exponential performance curve, not linear. It's just an absolute nightmare to try and amortize exponential performance curves with linear repayment.

For GB300 series, pricing along the lines of $500k/server, 9 servers/rack ($4.5M/rack) before looking at water cooling, 800Gb RDMA network, and your 100Gb IP network

There’s a risk that likely will happen that once demand is met capacity will overshoot significantly. However, when will that occur is very up in the air. Everyone is training every larger models and there’s more AI usage than ever

If Semi-Analyses is correct [1], even H100s would lose money at current prices - I doubt that is actually true of H100s in practice because from what I can tell it is based on a 3-year depreciation from the new price, but the point is that there is a point at which old hardware's operating inefficiencies dominate.

https://inferencex.semianalysis.com/infx/profit-estimator/de...

why? I keep seeing this over and over again but I've not seen an explanation. I would really appreciate some insight into this line of thinking, since it's not obvious to me. What is the benefit of holding hundreds of billions of computer hardware when it's not proven that it's even needed.

When AI replaces human workers and researchers, the "frontier", with which you can earn money and gain an edge over your competition, resides in those data centers with the highest concentration of computational power.

If you own those data centers, you control the frontier of human knowledge, the edge where money and power is being carved out.

Just simple napkin math.

Now, lets say there are 5 companies competing to be the winner. Only one can be the winner, but all 5 are trying to be the winner. How much should each company individually spend?

Well... starting-bid is $2 Trillion, or ~$10 Trillion in the aggregate. Its basically the waste of competition. Because everyone is vying for monopoly power, the only fair for them to all invest over $2 Trillion in the hopes of being the dominant one at the end. Even if $8 Trillion is wasted, that's "other companies", your competitors, who lose out.

But wait: if one company spends $2.1 Trillion, that one company likely becomes the winner and the other 4 companies lose out. So then another company spends $2.2 Trillion, to get ahead of that war. Etc. etc.

Yes.

> I have so many questions.

I'm not saying this is correct. I'm just saying a lot of other people believe this. If they are wrong then they will lose a lot of money. If they are correct, most of them will lose lots of money but one of them will make money.

I agree with you that the chance of "everyone is a loser" is higher than the current Zeitgeist believes.

The real gamble: betting there is a large enough pool of c-suite that doesn't fundamentally grasp the concept of vertical integration that they're willing to pay to train their competition. Because if you think AI companies aren't training models how to do everything your company does with your data you are deeply delusional.

The real real gamble: praying sovereign governments stay anesthitized to the national security implications of a handful of extra-national organizations operating the conputer security equivalent of WMD with no oversight.

or is the "winner takes all" mentality based on the assumption of being the first to achieve the technological singularity?

Alternatively, it may be based on becoming the favored provider of the defense or intelligence industries.

If you think that, then all this greed and stupidity makes sense.

If not, well, it's just greed and stupidity init.

There's plenty of data indicating that inference at typical API prices is profitable after accounting for the cost of GPUs. However that doesn't account for the arms race of training new models while performing R&D.