Every model learned by gradient descent is approximately a kernel machine (2020)

Re:Every model learned by gradient descent is approximately ... by aChattuio on Sunday February 25, 2024 @04:21AM (#39499153)

I argue that compression and not memorization is key to emerging behavior.

And the fact that llms learn languages and can switch quite well in-between (even if you just replace single words) is proof of learning meta/high level abstractions.

I sometimes write a German word or describe something when I don't have it on the tip of my tongue

[ Parent | on HN ]

Re:Every model learned by gradient descent is approximately ... by yard2010 on Sunday February 25, 2024 @05:13AM (#39499499)

Philosophically speaking, compression and memorization is how the human mind is learning

[ Parent | on HN ]