Re:Running large language models like ChatGPT on a single GPU
by tarr11 on Monday February 20, 2023 @01:24PM
(#34871017)
Wonder if someone would be willing to start an open source project where we could crowdsource donations for training, and people could possibly donate their GPU usage for it.
[ Parent
| on HN ]
Re:Running large language models like ChatGPT on a single GPU
by polishdude20 on Monday February 20, 2023 @01:37PM
(#34871178)
There's gotta be something like this already. Like a SETI @ Home type of thing .
Re:Running large language models like ChatGPT on a single GPU
by mryab on Monday February 20, 2023 @01:55PM
(#34871420)
There is! See https://petals.ml/ for inference of models like BLOOM-176B over the internet or https://arxiv.org/abs/2301.11913 and https://arxiv.org/abs/2206.01288 that show you how to do pretraining from scratch in the same setting. Disclaimer: I'm a coauthor of these systems (including the one in OP)
Re:Running large language models like ChatGPT on a single GPU
by exebook on Monday February 20, 2023 @03:26PM
(#34872464)
Amazing work! If I had a GPU, I'd join. I know similar project for text-to-image: https://aqualxx.github.io/stable-ui/
Re:Running large language models like ChatGPT on a single GPU
by efreak on Tuesday February 21, 2023 @09:59PM
(#34891080)
https://github.com/aqualxx/stable-horde-notebook
My only problem with stable horde is that their anti-cp measure involves checking the prompt for words like small, meaning I can't use a nsfw-capable model with certain prompts (holding a very small bag, etc). That, and seeing great things in the image rating and being unable to reproduce because it doesn't provide the prompt.