News for hackers. Stuff that matters.
Re:Machine learning works spectacularly well, but mathematic... by ilzmastr on Friday December 04, 2015 @09:15AM (#10676392)
Yea it's more like the opposite, in theory you only need 1 hidden layer to get the same capacity in the network as with more hidden layers, but in practice, since auto encoders, many hidden layers gradually decreasing in size are easier to get to from local energy minimums to globally low energy minimums
Re:Machine learning works spectacularly well, but mathematic... by crazypyro on Friday December 04, 2015 @09:40AM (#10676526)
Yes, people took the Universal approximation theorem[0] as evidence that they only need 1 layer, but there is zero guarantee of efficiency. A single hidden layer may mean many magnitudes more neurons needed over a n-hidden layer, n > 1 network, which could cause an unrealistic training time. Having multiple layers can reduce this training time with an optimal structure.

[0]https://en.wikipedia.org/wiki/Universal_approximation_theore...

Re:Machine learning works spectacularly well, but mathematic... by argonaut on Friday December 04, 2015 @01:16PM (#10677984)
Nobody ever took that theorem seriously. Deep nets were around since the 90s.
Re:Machine learning works spectacularly well, but mathematic... by crazypyro on Friday December 04, 2015 @03:43PM (#10679003)
I've seen people with passing knowledge of NNs throw it around sometimes and its also referred to in a lot of literature as one reason for needing to parallelize neural networks (which I've been reading a lot on, due to a project), even if its not hugely important.
Re:Machine learning works spectacularly well, but mathematic... by argonaut on Wednesday December 09, 2015 @06:23PM (#10707612)
It's not hugely important because it tells us little of practice use. I mean, k nearest neighbors, given infinite data, can model any function as well. In practice, single layer neural nets are not very useful and don't do a good job of learning feature representations.