Re:Do Deep Nets Really Need to be Deep? [pdf]
by oh_sigh on Saturday October 04, 2014 @06:24PM
(#8410840)
I thought it was already proven that a neural network with two hidden layers is as powerful as any neural network with an arbitrary number of layers?
[ Parent
| on HN ]
Re:Do Deep Nets Really Need to be Deep? [pdf]
by beagle3 on Saturday October 04, 2014 @06:37PM
(#8410866)
What I'm familiar with is described here:
http://neuron.eng.wayne.edu/tarek/MITbook/chap2/2_3.html
Basically, one hidden layer networks are already universal approximator; however, it's a nonconstructive result. This article is semi-constructive, - they train more complicated model, then emulate it with a simpler model, but do not know how to train the simple model directly.
Re:Do Deep Nets Really Need to be Deep? [pdf]
by ivan_ah on Saturday October 04, 2014 @06:30PM
(#8410853)
Right, but the narrative associated with this result is that the single-layer might require a lot more units to achieve the same performance as a deep architecture.
It seems this is not necessarily the case---based on the empirical results shown.