Re:Do Simpler Machine Learning Models Exist and How Can We F...
by tlarkworthy on Thursday December 22, 2022 @04:46PM
(#34099218)
All the books. I think books might be better
Re:Do Simpler Machine Learning Models Exist and How Can We F...
by recuter on Thursday December 22, 2022 @04:58PM
(#34099372)
There's only about 100m books (in English). About the same volume of text as the web all total.
Re:Do Simpler Machine Learning Models Exist and How Can We F...
by MonkeyClub on Thursday December 22, 2022 @05:56PM
(#34100050)
Generally a book has deeper thinking than a webpage, though, I think that’s the crux of the GP’s clarification.
Re:Do Simpler Machine Learning Models Exist and How Can We F...
by disgruntledphd2 on Friday December 23, 2022 @05:29AM
(#34104237)
And the distribution of text will be very different for all books vs the internet.
Re:Do Simpler Machine Learning Models Exist and How Can We F...
by albert_e on Thursday December 22, 2022 @11:57PM
(#34102468)
Yes.
And (with scary privacy implications) maybe the next frontier is capturing all spoken language uttered by people in real-time and streaming it into a model that is being updated in near-real-time.
And finally brain implants extracting unspoken thoughts and neural activity and combining it all. Extend it to non human life forms as well. (universal consciousness?)
Re:Do Simpler Machine Learning Models Exist and How Can We F...
by akomtu on Friday December 23, 2022 @03:06AM
(#34103393)
It's the future where everyone is a universal captcha solver for the needs of the Great AI. No freedom even in your thoughts.
Re:Do Simpler Machine Learning Models Exist and How Can We F...
by theGnuMe on Thursday December 22, 2022 @04:46PM
(#34099223)
> it's hard to see how they'll be able to significantly increase that.
With feedback.
Re:Do Simpler Machine Learning Models Exist and How Can We F...
by DenisM on Thursday December 22, 2022 @10:18PM
(#34102001)
Yes. Which is why ChatGPT is open to public.
Re:Do Simpler Machine Learning Models Exist and How Can We F...
by not2b on Thursday December 22, 2022 @05:13PM
(#34099556)
Perhaps some sort of adversarial network approach could work better; models that learn to generate text and other models that try to distinguish AIs from humans, competing against each other. Also, children learning language benefit from constant feedback from people who have their best interest at heart ... that last part is important because of episodes like Microsoft's Tay where 4chan folks thought it would be fun to turn the chatbot into a fascist.
Re:Do Simpler Machine Learning Models Exist and How Can We F...
by ronsor on Thursday December 22, 2022 @09:00PM
(#34101511)
I think OpenAI has already published some research showing humans preferred smaller/fewer parameter models that were "better trained" through the use of human feedback.
If there were a model that could adequately replace the role of the human, then that approach would probably work well.
Re:Do Simpler Machine Learning Models Exist and How Can We F...
by charcircuit on Thursday December 22, 2022 @11:20PM
(#34102308)
Passing the Turing test isn't the goal. You can have a useful model that isn't human like and can have a useless model that you can't tell isn't a human.
Re:Do Simpler Machine Learning Models Exist and How Can We F...
by not2b on Friday December 23, 2022 @03:16PM
(#34110138)
Certainly. Approaches that have done better on the Turing test have used various tricks to avoid their lack of understanding, like playing a paranoid person. But some of the best chatbots give themselves away by getting stuck in a loop or demonstrating lack of basic intuition about the world that a three year old would have. Those things perhaps could be caught.
Re:Do Simpler Machine Learning Models Exist and How Can We F...
by geysersam on Thursday December 22, 2022 @05:43PM
(#34099909)
The data is the bottleneck for the current generation of models. Better models/training strategies could very well change that in the next couple of decades.
Re:Do Simpler Machine Learning Models Exist and How Can We F...
by dr_dshiv on Thursday December 22, 2022 @06:10PM
(#34100195)
Well, diffusion networks seemed to work because they increased the amount of data by training so many examples of added noise and it’s removal. Some similar approach might be possible with text, too. Or, i guess they will talk to itself, like alpha go played itself.
People love to say "it's early" and "it will improve" about ChatGPT. but amount of training data IS the dominant factor in determining the quality of the output, usually in logarithmic terms. it's already trained on the entire internet. it's hard to see how they'll be able to significantly increase that.
And having models build models is drastically overrated. again, the accuracy/quality improvements are largely driven by the scale and diversity of the dataset. that's like 90% of the solution to any ml problem. choosing the right model and parameters is often a minor relative improvement