News for hackers. Stuff that matters.
Re:Machine learning works spectacularly well, but mathematic... by crazypyro on Friday December 04, 2015 @09:40AM (#10676526)
Yes, people took the Universal approximation theorem[0] as evidence that they only need 1 layer, but there is zero guarantee of efficiency. A single hidden layer may mean many magnitudes more neurons needed over a n-hidden layer, n > 1 network, which could cause an unrealistic training time. Having multiple layers can reduce this training time with an optimal structure.

[0]https://en.wikipedia.org/wiki/Universal_approximation_theore...

Re:Machine learning works spectacularly well, but mathematic... by argonaut on Friday December 04, 2015 @01:16PM (#10677984)
Nobody ever took that theorem seriously. Deep nets were around since the 90s.
Re:Machine learning works spectacularly well, but mathematic... by crazypyro on Friday December 04, 2015 @03:43PM (#10679003)
I've seen people with passing knowledge of NNs throw it around sometimes and its also referred to in a lot of literature as one reason for needing to parallelize neural networks (which I've been reading a lot on, due to a project), even if its not hugely important.
Re:Machine learning works spectacularly well, but mathematic... by argonaut on Wednesday December 09, 2015 @06:23PM (#10707612)
It's not hugely important because it tells us little of practice use. I mean, k nearest neighbors, given infinite data, can model any function as well. In practice, single layer neural nets are not very useful and don't do a good job of learning feature representations.