News for hackers. Stuff that matters.

A comment on Running large language models like ChatGPT on a single GPU. Parent comment

Re:Running large language models like ChatGPT on a single GPU by p1esk on Tuesday February 21, 2023 @10:36PM (#34891317)
8xH100 machine is ~300k I’ve heard.
[ Parent | on HN ]
Re:Running large language models like ChatGPT on a single GPU by lostmsu on Tuesday February 21, 2023 @11:07PM (#34891501)
Well, the person above claims 8xA100 significantly under $130k. I am curious to hear more.
[ Parent | on HN ]
Re:Running large language models like ChatGPT on a single GPU by p1esk on Wednesday February 22, 2023 @12:51AM (#34892140)
Sure, but you mentioned H100 machine, and those are about 2.5x more expensive.
[ Parent | on HN ]