Then how come you can still get several last gen EPYC or Xeon systems that would use the same amount of power for under $1 per hour?

For datacentre GPUs the energy, infrastructure and other variable costs seem to be relatively insignificant to fixed capital costs. Nvidia's GPUs are just extremely expensive relative to how much power they use (compared to CPUs).

> H100s you’re still better off renting them out for $2 per hour rather than having them idle at $0 per hour.

If you're barely breaking even at $2 then immediately selling them would seem like the only sensible option (depreciation alone is significantly higher than the cost power of running a H100 24x365 at 100% utilization).

> If you're barely breaking even at $2 then immediately selling them would seem like the only sensible option (depreciation alone is significantly higher than the cost power of running a H100 24x365 at 100% utilization).

If you can then probably yes. But why would someone else buy them (at the price you want), when they can rent at $2 per hour instead?

I don't think the why matters as long as people are buying them at very high prices, which they seemingly still are.

What makes you think they are?

Nvidia's quarterly income statements?

I'm not saying NVidia sales are slowing down (the books are complete for quite a while AIUI) but... Where would we hear first about a slowing down in sales? From a NVidia quarterly statement or from the market of renting GPU compute?

Used GPU prices still seem to be pretty high and availability is low? But yes, if the GPU compute rental market is highly unprofitable (I'm not sure it is, though) while HW prices are still high that indicates a clear inefficiency in the market. Meaning that you should sell ASAP before it self corrects.