News for hackers. Stuff that matters.
Re:Do Deep Nets Really Need to be Deep? [pdf] by oh_sigh on Saturday October 04, 2014 @06:24PM (#8410840)
I thought it was already proven that a neural network with two hidden layers is as powerful as any neural network with an arbitrary number of layers?
Re:Do Deep Nets Really Need to be Deep? [pdf] by beagle3 on Saturday October 04, 2014 @06:37PM (#8410866)
What I'm familiar with is described here: http://neuron.eng.wayne.edu/tarek/MITbook/chap2/2_3.html

Basically, one hidden layer networks are already universal approximator; however, it's a nonconstructive result. This article is semi-constructive, - they train more complicated model, then emulate it with a simpler model, but do not know how to train the simple model directly.

Re:Do Deep Nets Really Need to be Deep? [pdf] by ivan_ah on Saturday October 04, 2014 @06:30PM (#8410853)
Right, but the narrative associated with this result is that the single-layer might require a lot more units to achieve the same performance as a deep architecture.

It seems this is not necessarily the case---based on the empirical results shown.