News for hackers. Stuff that matters.
Re:DeepMath Conference 2020 – Conference on the Mathematical... by Der_Einzige on Saturday November 07, 2020 @01:02PM (#25016901)
While in practice this is true, a lot of work is being done to try to figure out how to make shallow neural networks train as fast as deep ones. We know by the UAT that in principle a shallow network with similar paramaters counts to a deep one should be capable of learning the same decision boundary...

https://en.m.wikipedia.org/wiki/Universal_approximation_theo...

Re:DeepMath Conference 2020 – Conference on the Mathematical... by aborsy on Saturday November 07, 2020 @01:11PM (#25017013)
It depends on the function that you are trying to approximate.

I can give you a function that a shallow nnet would approximate better and functions that deep nets approximate exponentially better even with one more layer (in terms of number of neurons n). In the limit n->\infty, both reach arbitrary small errors (obviously often with different number of parameters).

Re:DeepMath Conference 2020 – Conference on the Mathematical... by falcor84 on Saturday November 07, 2020 @01:37PM (#25017267)
Is there a particular theorem that you're invoking here?