News for hackers. Stuff that matters.

A comment on $2 H100s: How the GPU Rental Bubble Burst.

Re:$2 H100s: How the GPU Rental Bubble Burst by hislaziness on Thursday October 10, 2024 @10:49PM (#41805550)
TLDR: Don’t buy H100s. The market has flipped from shortage ($8/hr) to oversupplied ($2/hr), because of reserved compute resales, open model finetuning, and decline in new foundation model co’s. Rent instead.

Is the AI infra bubble already bursting?

[ Parent | on HN ]
Re:$2 H100s: How the GPU Rental Bubble Burst by pico_creator on Thursday October 10, 2024 @10:56PM (#41805580)
I’m hopping more for an open weights AI boom

With cheap compute for everyone to finetune :)

[ Parent | on HN ]
Re:$2 H100s: How the GPU Rental Bubble Burst by TechDebtDevin on Thursday October 10, 2024 @10:55PM (#41805576)
No, but the prices will likely converge with MSRP pricing. A lot of datacenter were filled with h100s that cost a premium to get ahold of.
[ Parent | on HN ]
Re:$2 H100s: How the GPU Rental Bubble Burst by hislaziness on Friday October 11, 2024 @12:03AM (#41805991)
It is not just MSRP, management and operations cost too. The article goes into the details of this.
[ Parent | on HN ]
Re:$2 H100s: How the GPU Rental Bubble Burst by pico_creator on Friday October 11, 2024 @02:09AM (#41806685)
Q_Q yes - ur right on that - and i wrote the article (about a month ago)
[ Parent | on HN ]
Re:$2 H100s: How the GPU Rental Bubble Burst by pico_creator on Thursday October 10, 2024 @10:57PM (#41805584)
Covered in the article. They are below MSRP essentially
[ Parent | on HN ]
Re:$2 H100s: How the GPU Rental Bubble Burst by justahuman74 on Friday October 11, 2024 @12:55AM (#41806300)
Yes, please only rent instead

- sincerely, all of the cloud providers

[ Parent | on HN ]
Re:$2 H100s: How the GPU Rental Bubble Burst by pico_creator on Friday October 11, 2024 @02:05AM (#41806660)
~Cough~ not all cloud provider (there are many still willing to charge you an arm and a leg)

Only the ones who can give you below MSRP essentially

[ Parent | on HN ]
Re:$2 H100s: How the GPU Rental Bubble Burst by Sohcahtoa82 on Friday October 11, 2024 @03:15PM (#41812575)
An H100 is what, $50,000 MSRP?

At $2/hr, that's 2.8 years to RoI. And that's just for the GPU and not the other hardware you'll need to plug it into, and doesn't include the power, and also assumes you're using it 100% of the time. Really, you're probably looking at 3.5+ years to RoI.

I'd rather rent than buy in that scenario.

[ Parent | on HN ]
Re:$2 H100s: How the GPU Rental Bubble Burst by swyx on Friday October 11, 2024 @02:50AM (#41806943)
(editor here) we've been commenting on the Winds of AI Winter for a while now :) https://latent.space/p/mar-jun-2024
[ Parent | on HN ]