That's because there's a temporary supply side crunch.

That only explains why the price of those GPUs at resale is high, but it doesn't invalidate the fact that those GPUs are still very useful way past a 2 to 3 years depreciation schedule advocated by Michael Burry et al.

Older GPUs can still be used to run model inference, albeit more slowly than newer GPUs. We already see frontier labs turning this insight into an opportunity to charge a premium on token delivery speed, when models run on newer hardware. As AI becomes more widely spread throughout the economy, not all workloads will require fast inference, and there will be many use cases for previous generation hardware. The only exception is if the latest batch of specialised chips become ubiquitous very quickly, but then price of compute AND intelligence per watt will both go down dramatically, which would reduce the need for sustained high capex and opex

Yes it is a supply-side crunch (chip manufacturing), but IMHO, it is not temporary. Only two things will remove the supply constraint.

- Demand drops like a stone: raw TFLOPS / per model. This will happen if someone figures out insanely fast and power-efficient math (specifically, multiplication) for existing chip architectures, that makes it dirt-cheap to keep re-computing ever-better SOTA LLMs, as well as to on-line compute inference with ever-bigger models (scaling laws rule). These are different workloads, AFAIK.

- Moore's law re-starts, because someone figures out entirely new transistor designs that can deliver at least up to 100x per-core frequency gains over Si transistors (which would be nothing considering that Si transistor tech delivered ~1Mx gains since invention) and which can be manufactured at similar bin sizes, at similar scale (i.e. equivalent per-die yield), price-competitively (no more 10x the equivalent in SOTA Silicon compute).

I'd say that a math breakthrough might be more accessible than the transistor-manufacturing breakthrough.

> Moore's law re-starts, because someone figures out entirely new transistor designs that can deliver at least up to 100x per-core frequency gains over Si transistors (which would be nothing considering that Si transistor tech delivered ~1Mx gains since invention)

That assumes that there is no hard, physical limit on Moore's Law, which is obviously false. There is a limit, and as long as we depend on mass flow of charge through conductors, it's nowhere near quantum sizes.

Where that limit is has not yet been determined, but at this point the support structure to write transistors is the size of a house trailer, using a staggering amount of toxic chemicals (just to run the exotic lasers). So, advancements today may scale very nonlinearly in cost.

The very phrase "Moore's law restarts" acknowledges that we are no longer on-track.

Yep, I mean to imply practically impossible, in context of bubble-timelines. Theoretically anything is possible given enough decades. Practically, AGI has thus far been 20 years out, as has "clean" fusion power, 1 gee EM drives etc.

> Theoretically anything is possible given enough decades.

No, theoretically Moore's Law has a very hard limit, regardless of time.

IDK if Moore's law restarting is possible. But I wouldn't rule out some sort of non-silicon compute breakthrough that functionally jumps us a couple orders of magnitude. The entire world economy being aggressively focused on compute seems like a good environment to drive investment into figuring out photonic computers or whatever. We do know at least one other non silicon highly energy efficient compute technology (biological brain) exists and works very well

I reckon it depends on how hot they run GPUs for how long. Running any chip at clock or overclocked (likely) will burn it out faster than "business as usual" workloads.

Defraying of capex also depends on how the hyperscalers scrap used hardware.

AFAIK, all hyperscaler hardware is effectively a consumable. Disks, memory, boards, chips; all the high-value pieces do not get refurbished into new computers for sale into secondary markets. The commercial and legal liability is too high---leaking their own proprietary IP as well as leaking customer IP, not to mention secrets and hardening configs and so forth.

I can't recall where I read it, but if a datacenter is modeled as a bunch of black-box hot-swappable shipping containers, then for pre-LLM workloads, per-container unit economics works only as long as a container delivers over 30% capacity. After that you got to swap it out, and send the black box into a furnace to recover them rare earth metals for the next generation of chips.

To a first approximation, high-throughput, gpu-maxxing math workloads (LLM inference) are surely harder on chips and memory, than are bread-and-butter SaaS / data lake / ETL / RDBMS workloads. Speculating further, I'd guess that the ~40% operating margin of datacenters is due to the relative compute-friendliness of non-math workloads for lack of better words.

> Running any chip at clock or overclocked (likely) will burn it out faster than "business as usual" workloads.

Could it be that they _have_ to run them like that because of the shortage of compute volume and once there are sufficient data centers, they can be run relatively cooler and thereby extending their lifespans? That doesn't bode well for NVIDIA/AMD.

> AFAIK, all hyperscaler hardware is effectively a consumable. Disks, memory, boards, chips; all the high-value pieces do not get refurbished into new computers for sale into secondary markets. The commercial and legal liability is too high---leaking their own proprietary IP as well as leaking customer IP, not to mention secrets and hardening configs and so forth.

Maybe that could be a catalyst for going for more "commodity" approach to hardware instead of highly spcialized (atleast for some sections of the hardware) so that the IP is not the hardware but the model/software and they can re-coup the costs by selling the hardware to "tier-2" providers that might emerge based on using cheaper second-hand hardware?

I suspect GPU and CPU failure rates have more to do with stress from thermal cycling, if they can keep them loaded and running without heating/cooling cycles, they should last pretty long. Their lifetime is probably dictated more from when do they stop being competitive on a flop/watt basis than by their failure rate.

Interesting... Upon reflection, I feel like it's a "yes, and" proposition coming down to failure mode.

[Type A] Constant use a constant power level: Quantum world rules would dominate.

When run hot, electron tunneling will kill off individual transistors, and AFAIK that is a cascading failure mode once it takes root. A big enough failed spot, even if localised, could effectively kill a core (if not the whole chip---too many errors). A local hotspot will directly affect neighbours, and indirectly the chip as a whole because the same work is being pushed on fewer cores.

[Type B] Thermal cycled: Mechanical and electrical factors would dominate...

Microfractures / mechanical damage in weak areas of whole chips, and motherboards. In a hyperscaler server rack, if a board dies that part of the rack is dead---micro-maintenance is probably too expensive.

And speculating further, it stands to reason that Type B failures would likely cause Type A failures, and vice-versa.

> Running any chip at clock or overclocked (likely) will burn it out faster

server parts are tuned for power efficiency and longevity

B200 has an operating range of 10-35C. https://resources.nvidia.com/en-us-dgx-systems/dgx-b200-data...

RTX 5090 (and most consumer CPU/GPUs) have thermal limits at 90-100C.

10-35 would be inlet or ambient temps, not what the actual dies or packages are expected to run at.

> The $2 for every $1 inference cost quote from Anthropic I presume includes all opex

No, it does not, its just revenue minus compute. Per their S1, Anthropic has never turned an operational profit.

That's simply not true. See page 160 of the S-1 - clearly shows positive operating profit in Q2 2026.

> And have good resale value for other use cases.

Do they? Is H200 $-competitive with an ASIC for crypto? Bearing in mind they're 4 figures.

Nobody cares about crypto anymore.

Neural networks in general are here to stay. Whether that is the current day frontier-model style LLM or not doesn't really matter. We've already had multiple different Deep Learning/Machine Learning architectures that have been incredibly applicable to multiple different domains.

Especially right now in the RAM-pocalpyse where it is difficult to get any machine, yes, these older GPUs are still very very useful. A guy at my company was having trouble getting compute allocated, budgeted, and approved - so eventually he went and dug up a bunch of old Volta-based GPUs, setup a local compute farm, and is getting work done. They are absolutely slower than current state-of-the-art, but they are still better than no compute at all.

I think one of the silver-linings of the current hardware high prices and shortages, is that people are going to (A) focus on software efficiency (B) not be so obsessed with having the latest and greatest hardware. Even on the consumer side, how many games ACTUALLY can utilize a RTX 5090?