News for hackers. Stuff that matters.
Re:Do Deep Nets Really Need to be Deep? [pdf] by wodenokoto on Monday October 06, 2014 @01:01AM (#8414353)
No, shallow NN definitely overfits in normal circumstances. However, in this case they are building a shallow NN that is not learning a normal function, but is being trained to emulate a deep network of equivalent amount of parameters.

Parents claim is that in that case, we don't have to worry about over fitting, since we want every result of target network to be as closely emulated as possible in the shallow network.

There are no comments on this comment yet.