News for hackers. Stuff that matters.

A comment on Running large language models like ChatGPT on a single GPU. Parent comment

Re:Running large language models like ChatGPT on a single GPU by anononaut on Monday February 20, 2023 @03:25PM (#34872460)
I heard it was $4MM alone in AWS compute time.
[ Parent | on HN ]
Re:Running large language models like ChatGPT on a single GPU by speedgoose on Monday February 20, 2023 @03:31PM (#34872530)
This number seems to match the $200k, if you take into account the cloud margins of our favourite counterfeit products reseller.
[ Parent | on HN ]
Re:Running large language models like ChatGPT on a single GPU by postalrat on Monday February 20, 2023 @03:30PM (#34872520)
So about 40k on your own machines.
[ Parent | on HN ]
Re:Running large language models like ChatGPT on a single GPU by inciampati on Monday February 20, 2023 @04:34PM (#34873237)
It does make it seem like a box of H100s will easily be able to make an interesting open LLM.
[ Parent | on HN ]