News for hackers. Stuff that matters.

A comment on Google DeepMind CEO Says Some Form of AGI Possible in a Few Years. Parent comment

Re:Google DeepMind CEO Says Some Form of AGI Possible in a F... by meindnoch on Wednesday May 03, 2023 @08:16AM (#35801422)
Well, yes. The 'magic' is the nonlinearity. It's because the composition of linear (actually also affine) functions is still just linear (affine). You don't get any additional power by combining many of them - which is also the reason why linear functions are so well understood and easy to work with.

You give sprinkle in a tiny nonlinearity (e.g. x^2 instead of x) and suddenly you can get infinite complexity by weighted composition - which is also the reason why we're so helpless with nonlinear functions and reach for linear approximations immediately (cf. gradient descent).

[ Parent | on HN ]
Re:Google DeepMind CEO Says Some Form of AGI Possible in a F... by imtringued on Thursday May 04, 2023 @02:22AM (#35811830)
>It's because the composition of linear (actually also affine) functions is still just linear (affine).

Except you can "compose" affine functions using Horner's schema to get any polynomial... No need to sprinkle tiny non linearities. It's a buffet. Grab as much as you need.

[ Parent | on HN ]