Re:A visual proof that neural nets can compute any function
by Houshalter on Wednesday September 03, 2014 @01:39AM
(#8261654)
It can learn any input to output mapping. What more do you want?
Re:A visual proof that neural nets can compute any function
by mtdewcmu on Thursday September 04, 2014 @02:08AM
(#8266991)
The article doesn't say anything about whether it can learn it, just that it can represent it.
Re:A visual proof that neural nets can compute any function
by Houshalter on Thursday September 04, 2014 @06:31PM
(#8271036)
The proof is constructive, so actually you can. Just create a few neurons for the "tower function" and the correct output weight at every point you want to map.
If you mean learn functions efficiently with few parameters, or in ways that generalize well, the article makes no claims about that at all. There are two "no free lunch" theorems. One of which says that it's impossible to guarantee any method will generalize well on all problems, and the other that says it's impossible to guarantee any optimization method will work well on all problems. You just have to make strong assumptions like "my function can be modeled by a neural net" or "the error function is convex."
However neural networks are agnostic to the optimization algorithm you use to set their weights. There are many different ones.
Later on in the article, it's qualified, but tempers are already rising.
Here come people with their non-computable functions, their unmeasurable functions, their nowhere-continuous functions, all wanting to get approximated. In sup norm! On unbounded sets!
Irksome, greedy, and vexatious.