News for hackers. Stuff that matters.

A comment on A visual proof that neural nets can approximate any function.

Re:A visual proof that neural nets can approximate any function by uuwp on Sunday April 21, 2019 @04:27PM (#19714366)
This proof is largely irrelevant in the real world. An interesting question would be how much can be approximated with a model that has 1 MB worth of weights and can use only relu/tanh/softmax activations.
[ Parent | on HN ]
Re:A visual proof that neural nets can approximate any function by darkkindness on Sunday April 21, 2019 @06:29PM (#19714991)
The first paragraph of the conclusion addresses that this is merely a proof of what is possible, not what is practical:

> The explanation for universality we've discussed is certainly not a practical prescription for how to compute using neural networks! In this, it's much like proofs of universality for NAND gates and the like. For this reason, I've focused mostly on trying to make the construction clear and easy to follow, and not on optimizing the details of the construction. However, you may find it a fun and instructive exercise to see if you can improve the construction.

[ Parent | on HN ]