News for hackers. Stuff that matters.
Re:Gato – A Generalist Agent by abriosi on Tuesday May 17, 2022 @07:20PM (#31417389)
Transformers managed to confuse me about what interpolation and extrapolation really mean in high dimensional space
Re:Gato – A Generalist Agent by abel_ on Tuesday May 17, 2022 @08:23PM (#31417808)
There's a neat argument against these models doing interpolation: the manifold of the data is so sparse that it's infinitesimally unlikely for a good predictor to be doing interpolation between existing points on the manifold.

https://arxiv.org/abs/2110.09485

Re:Gato – A Generalist Agent by asdfman123 on Tuesday May 17, 2022 @07:27PM (#31417444)
Yes, nice, I know some of these words
Re:Gato – A Generalist Agent by qrian on Tuesday May 17, 2022 @07:45PM (#31417552)
Maybe what transformers actually transform are us..
Re:Gato – A Generalist Agent by moffkalast on Tuesday May 17, 2022 @07:37PM (#31417503)
Autobots, print out.