Re:A visual proof that neural nets can compute any function
by szabba on Wednesday September 03, 2014 @03:08PM
(#8264694)
I'm pretty sure linearity was also part of the picture -- no matter how many layers of neurons you have, as long as they perform linear combinations, they could all be replaced with a single one (ignoring numeric errors).
[ Parent
| on HN ]
Re:A visual proof that neural nets can compute any function
by jo_ on Wednesday September 03, 2014 @03:20PM
(#8264766)
I mentioned this in my post ("given nonlinear activations"), but you're correct to point out its importance. The combination of linear functions is always linear. Full stop. You need at least one non-linear layer to get reasonable results.