News for hackers. Stuff that matters.

A comment on Running large language models like ChatGPT on a single GPU. Parent comment

Re:Running large language models like ChatGPT on a single GPU by lostmsu on Tuesday February 21, 2023 @06:39PM (#34889368)
> We got our "deal" when buying just a single server and have since bought many more with the same provider. We didn't spend 7 figures all at once, we did it piece-meal over time. There is nothing stopping you from getting much better prices.

If it is as easy as you make it sound, why would you not just share the vendor name? I personally would love an 8xH100 machine for transformer experiments, but $100k+ pricing makes it a non-starter.

[ Parent | on HN ]
Re:Running large language models like ChatGPT on a single GPU by p1esk on Tuesday February 21, 2023 @10:36PM (#34891317)
8xH100 machine is ~300k I’ve heard.
[ Parent | on HN ]
Re:Running large language models like ChatGPT on a single GPU by lostmsu on Tuesday February 21, 2023 @11:07PM (#34891501)
Well, the person above claims 8xA100 significantly under $130k. I am curious to hear more.
[ Parent | on HN ]
Re:Running large language models like ChatGPT on a single GPU by p1esk on Wednesday February 22, 2023 @12:51AM (#34892140)
Sure, but you mentioned H100 machine, and those are about 2.5x more expensive.
[ Parent | on HN ]
Re:Running large language models like ChatGPT on a single GPU by cardine on Wednesday February 22, 2023 @05:13PM (#34903695)
The contract has very clear language not to share it - likely because they are offering different prices to different companies.

(And as p1esk mentioned, there is no way you are getting H100s for under $100k).

[ Parent | on HN ]