News for hackers. Stuff that matters.
Re:Do Deep Nets Really Need to be Deep? [pdf] by fredophile on Sunday October 05, 2014 @01:16AM (#8411510)
I skimmed the paper so I may have missed this but is there a benefit to using a shallow network to simulate the deep network? While this is an interesting result I'd think that for now if I already have a trained deep network I'd just use it instead of training a shallow network to mimic it.
Re:Do Deep Nets Really Need to be Deep? [pdf] by Houshalter on Sunday October 05, 2014 @01:51PM (#8412876)
While it doesn't have to be shallow, there is an advantage in training a smaller model to mimic a more complicated one. The smaller net uses less computations.

Neural nets are often trained to be as large as possible on clusters of GPUs. Machine learning also commonly uses ensembles of dozens of different models. So there is a lot to be gained by compressing it down to a single model.