Re:Gato – A Generalist Agent
by sva_ on Tuesday May 17, 2022 @06:23PM
(#31416978)
> because we've already seen where scaling these mechanisms leads.
In the case of GPT-3, scaling seemed to continuously improve results, they just kinda ran out of data. Are you implying this must be the same for this model? Or were you intending to say something different that I didn't see?
Re:Gato – A Generalist Agent
by mach1ne on Wednesday May 18, 2022 @03:32AM
(#31420147)
To be precise, scaling improved results logarithmically. Accuracy seems to plateau with scaling and never reach any magical limit of true general intelligence.
In the case of GPT-3, scaling seemed to continuously improve results, they just kinda ran out of data. Are you implying this must be the same for this model? Or were you intending to say something different that I didn't see?