News for hackers. Stuff that matters.
Re:Do Deep Nets Really Need to be Deep? [pdf] by Animats on Saturday October 04, 2014 @05:06PM (#8410655)
This is fascinating. "Model compression demonstrates that a small neural net could, in principle, learn the more accurate function, but with the current learning algorithms we are unable to train a model with that accuracy on the original training data; instead, we must train the complex ensemble first and then train the neural net to mimic it."

That's a profound result. It bears on how "abstraction" works, and gives some insight on how learned information becomes more rigid. Once you've compressed the neural net, you can't update the compressed form, which has much less state, based on new data.

Re:Do Deep Nets Really Need to be Deep? [pdf] by jostmey on Saturday October 04, 2014 @05:11PM (#8410672)
Read the last line of the abstract. It sounds like there is more to follow, and that this extended abstract is the tip of an iceberg.