News for hackers. Stuff that matters.

Ask HN: How bad is the GPU shortage for startups/researchers?

Curious how folks are faring with GPU access for training / inference? (those of us outside of big contracts / commits with the big three)

I've seen folks try to address the shortage through a number of mechanisms: - Co-op pool for recipients of a specific AI investment fund - Collating a transparent price book (Shadeform) - People putting a rack under their desks?

(Full disclosure, I am thinking about renting out GPUs, trying to get a pulse on the market)

( 4 comments | 12 points | on HN )
Re:Ask HN: How bad is the GPU shortage for startups/research... by steerex on Wednesday August 23, 2023 @07:21PM (#37243224)
I haven't actually needed access so haven't tested this yet, but a month ago an AWS rep told me that priority went to folks who were willing to make up-front commitments; the longer-term, the better. He implied that there was a serious shortage.
Re:Ask HN: How bad is the GPU shortage for startups/research... by getwiththeprog on Thursday August 24, 2023 @02:38AM (#37245743)
car salesman told me they weren't getting any new cars so I should buy now
Re:Ask HN: How bad is the GPU shortage for startups/research... by quickthrower2 on Wednesday August 23, 2023 @08:14PM (#37243598)
For hobby stuff: no problem. VastAI. I imagine the issue is when you need banks of 8xA100s machines.
Re:Ask HN: How bad is the GPU shortage for startups/research... by rkwasny on Thursday August 24, 2023 @02:12AM (#37245568)
If you need 8xA100 you can get them cheaply on GCP, just not in any region.

If you need 8xH100 only option is AWS, p5.48xlarge - $98/hour

I recommend skypilot for finding a good region