News for hackers. Stuff that matters.
Re:Gated Linear Networks by contravariant on Tuesday June 16, 2020 @06:23AM (#23537518)
Universal approximation doesn't seem that strong a claim at all, you could use simple polynomials to achieve just that (to some extent RELUs are polynomials as well, provided you use 0/1 or integer weights, but that's probably besides the point).

It would be much weirder for a function with lots of parameters to not have universal approximation in some sense, as it would imply that you lose some degrees of freedom somewhere.

There are no comments on this comment yet.