News for hackers. Stuff that matters.

A comment on Running large language models like ChatGPT on a single GPU. Parent comment

Re:Running large language models like ChatGPT on a single GPU by neilmovva on Monday February 20, 2023 @03:41PM (#34872631)
While OPT-175B is great to have publicly available, it needs a lot more training to achieve good results. Meta trained OPT on 180B tokens, compared to 300B that GPT-3 saw. And the Chinchilla scaling laws suggest that almost 4T tokens would be required to get the most bang for compute buck.

And on top of that, there are some questions on the quality of open source data (The Pile) vs OpenAI’s proprietary dataset, which they seem to have spent a lot of effort cleaning. So: open source models are probably data-constrained, in both quantity and quality.

[ Parent | on HN ]
Re:Running large language models like ChatGPT on a single GPU by mensetmanusman on Monday February 20, 2023 @03:50PM (#34872734)
It’s fun to think about a few billion weights being the difference between useless and gold.
[ Parent | on HN ]
Re:Running large language models like ChatGPT on a single GPU by tiborsaas on Monday February 20, 2023 @04:17PM (#34873042)
Looking at my bank account I can relate :)
[ Parent | on HN ]
Re:Running large language models like ChatGPT on a single GPU by Miraste on Monday February 20, 2023 @08:49PM (#34875481)
OPT-175B isn't publicly available, sadly. It's available to research institutions, which is much better than "Open"AI, but it doesn't help us hobbyists/indie researchers much.
[ Parent | on HN ]
Re:Running large language models like ChatGPT on a single GPU by generalizations on Tuesday February 21, 2023 @08:50AM (#34881093)
I wonder when we'll start putting these models on the pirate bay or similar. Seems like an excellent use for the tech. Has no one tried to upload OPT-175B anywhere like that yet?
[ Parent | on HN ]
Re:Running large language models like ChatGPT on a single GPU by cma on Tuesday February 21, 2023 @01:23PM (#34884737)
It could go on the clear net since trained weights aren't subject to copyright.
[ Parent | on HN ]