Re:Introducing ChatGPT Plus
by sterlind on Wednesday February 01, 2023 @02:42PM
(#34616415)
why can't crowdfunding work for this stuff? I'd gladly chip in like, $1K or something, to fund the training of a ChatGPT-like LLM, on the condition that it's publicly released with no fetters.
[ Parent
| on HN ]
Seems like you would have to declare an entity to receive funds which is a no-no if you’re setting out to do something illegal.
It's not illegal yet to train an LLM. Best to get started before they lock it down and entrench the monopolies.
We are currently at "mainframe" level of AI. It takes a room sized computer and millions of dollars to train a SOTA LLM.
Current models are extremely inefficient, insofar as they require vast internet-sized data, yet clearly we have not gotten fully human-quality reasoning out. I don't know about you, but I didn't read the entire Common Crawl in school when I was learning English.
The fundamental bottleneck right now is efficiency. ChatGPT is nice as an existence proof, but we are reaching a limit to how big these things can get. Model size is going to peak and then go down (this may already have happened).
So while we could crowdfund a ChatGPT at great expense right now, it's probably better to wait a few years for the technology to mature further.