If they don't burn out after two or three. Annualized failure rates are about 9% according to Meta.
Less than half of the original hardware will be running after ten years even if you assume that rate does not increase over time. Do H100s come with replacement guarantees?
This paper puts the failure rate of A100s closer to 0.3%/yr, H100s under 1%.
Mind you, most of the H100s that have ever been sold are still in their packaging, I suspect. Or at least not plugged in.
> [...] because quite frankly, the biggest issue we are now having is not a compute glut, but it’s power — it’s sort of the ability to get the builds done fast enough close to power. So, if you can’t do that, you may actually have a bunch of chips sitting in inventory that I can’t plug in. In fact, that is my problem today. It’s not a supply issue of chips; it’s actually the fact that I don’t have warm shells to plug into.
https://www.tomshardware.com/tech-industry/artificial-intell...
I do agree that 5 years isn't the hard cutoff I was suggesting but I'm not convinced that 10 years is a reasonable timeframe to consider paying the debt down when Moore's Law is involved. I know it's no longer the 12-18mo doubling it used to be but even if it's 2ish years then over 6 years you get 3 doublings or 8x performance gains. At 10 years it's another 2 doublings or 32x performance gains. It's hard for linear payback curves to deal with exponential performance curves past 2-3 doublings.
IF prices of GPU/RAM come back to pre bubble costs, we might see the 5 year cycle of replacement that's relatively common in normal datacenters pre-ai bubble.
At current prices ? those GPUs that are there now will be running there in 10 years still.