Re:Gato – A Generalist Agent
by abriosi on Tuesday May 17, 2022 @07:20PM
(#31417389)
Transformers managed to confuse me about what interpolation and extrapolation really mean in high dimensional space
[ Parent
| on HN ]
There's a neat argument against these models doing interpolation: the manifold of the data is so sparse that it's infinitesimally unlikely for a good predictor to be doing interpolation between existing points on the manifold.
Re:Gato – A Generalist Agent
by asdfman123 on Tuesday May 17, 2022 @07:27PM
(#31417444)
Yes, nice, I know some of these words
Maybe what transformers actually transform are us..
Re:Gato – A Generalist Agent
by moffkalast on Tuesday May 17, 2022 @07:37PM
(#31417503)
Autobots, print out.