News for hackers. Stuff that matters.

A comment on Could you train a ChatGPT-beating model for $85,000 and run it in a browser?.

Re:Could you train a ChatGPT-beating model for $85,000 and r... by agnokapathetic on Friday March 31, 2023 @02:31PM (#35391279)
> My friends at Replicate told me that a simple rule of thumb for A100 cloud costs is $1/hour.

AWS charges $32/hr for an 8xA100s (p4d.24xlarge) which comes out to $4/hour/gpu. Yes you can get lower pricing with a 3 year reservation but thats not what this question is asking.

You also need 256 nodes to be colocated on the same fabric -- which AWS will do for you but only if you reserve for years.

[ Parent | on HN ]
Re:Could you train a ChatGPT-beating model for $85,000 and r... by thewataccount on Friday March 31, 2023 @02:38PM (#35391377)
AWS certainly isn't the cheapest for this, did they mention using AWS? Lamdba Labs is 12$/hr for 8xA100's, and there's others relatively close to this price on demand, I assume you can get a better deal if you contact them for a large project.

Replicate themselves rent out GPU time so I assume they would definitely know as that's almost certainly the core of their business.

[ Parent | on HN ]
Re:Could you train a ChatGPT-beating model for $85,000 and r... by celestialcheese on Friday March 31, 2023 @02:41PM (#35391419)
lambdalabs will let you do on-demand 8xa100 @ 80GB VRAM/GPU for $12/hr, or reserved @ $10.86/hr

8xA100 @ 40gb for $8/hr

Replicate friend isn't far off.

[ Parent | on HN ]
Re:Could you train a ChatGPT-beating model for $85,000 and r... by IanCal on Friday March 31, 2023 @02:38PM (#35391382)
Lambda labs charges about 11-12/hr for 8xA100.
[ Parent | on HN ]
Re:Could you train a ChatGPT-beating model for $85,000 and r... by robmsmt on Friday March 31, 2023 @02:45PM (#35391481)
and is completely at capacity
[ Parent | on HN ]
Re:Could you train a ChatGPT-beating model for $85,000 and r... by IanCal on Friday March 31, 2023 @05:13PM (#35393747)
But reflects an upper bound at the cost of running a100s.
[ Parent | on HN ]
Re:Could you train a ChatGPT-beating model for $85,000 and r... by sebzim4500 on Friday March 31, 2023 @02:35PM (#35391335)
Maybe they are using spot instances? $1/hr is about right for those.
[ Parent | on HN ]
Re:Could you train a ChatGPT-beating model for $85,000 and r... by pavelstoev on Friday March 31, 2023 @05:04PM (#35393635)
model-depending, you can train on lesser (cheaper) GPUs but system-level optimizations are needed. Which is what we provide at centml.ai
[ Parent | on HN ]