News for hackers. Stuff that matters.

A comment on Empiricism and the limits of gradient descent. Parent comment

Re:Empiricism and the limits of gradient descent by simonster on Monday May 28, 2018 @01:04AM (#17170677)
It's a fair point that the Universal Approximation Theorem does not guarantee that the weights can be learned. OTOH, the physical laws that the article states a neural network cannot discover are computable functions.
[ Parent | on HN ]
Re:Empiricism and the limits of gradient descent by AstralStorm on Monday May 28, 2018 @03:48AM (#17171139)
You need a stronger bound than this. They have to be possible to approximate govern specific network size, architecture and activation functions. Calculating that (or good statistics that will say so approximately) is a hard problem... It is solvable for a bunch of activations in a layered perceptron but attempt extending this to something more complex.
[ Parent | on HN ]