Running large language models like ChatGPT on a single GPU

Re:Running large language models like ChatGPT on a single GPU by leesec on Monday February 20, 2023 @03:22PM (#34872414)

No it isn't. Stable Diffusion is less than 200 grand to train.

[ Parent | on HN ]

Re:Running large language models like ChatGPT on a single GPU by huijzer on Monday February 20, 2023 @03:26PM (#34872465)

According to Christopher Potts (Stanford Professor and Chair, Department of Linguistics, and Professor, by courtesy, Department of Computer Science), training a large language model costs about 50 million [1].

[1]: https://youtu.be/-lnHHWRCDGk?t=637

[ Parent | on HN ]

Re:Running large language models like ChatGPT on a single GPU by nl on Monday February 20, 2023 @07:48PM (#34874987)

Yeah this is way wrong, unless counting the salaries of everyone involved for a few years in the lead up while writing software that ended up being used

https://bigscience.huggingface.co/blog/bloom was trained with a $7M grant, and that was the first time they'd done it (so there was a lot of failures).

[ Parent | on HN ]

Re:Running large language models like ChatGPT on a single GPU by anononaut on Monday February 20, 2023 @03:25PM (#34872460)

I heard it was $4MM alone in AWS compute time.

[ Parent | on HN ]

Re:Running large language models like ChatGPT on a single GPU by speedgoose on Monday February 20, 2023 @03:31PM (#34872530)

This number seems to match the $200k, if you take into account the cloud margins of our favourite counterfeit products reseller.

[ Parent | on HN ]

Re:Running large language models like ChatGPT on a single GPU by postalrat on Monday February 20, 2023 @03:30PM (#34872520)

So about 40k on your own machines.

[ Parent | on HN ]

Re:Running large language models like ChatGPT on a single GPU by inciampati on Monday February 20, 2023 @04:34PM (#34873237)

It does make it seem like a box of H100s will easily be able to make an interesting open LLM.

[ Parent | on HN ]