News for hackers. Stuff that matters.
Re:The Unreasonable Effectiveness of Recurrent Neural Networks by dwiel on Thursday May 21, 2015 @11:19PM (#9586449)
There are some recent examples where people have trained a collection of large nets which are then used to teach a smaller net. The smaller net can learn more quickly and finally achieves better performance than the large collection.

The methods involve providing more detailed feedback at each example. With most training data used now, we give a 0 or 1, does this example belong to this class. In the teacher networks, they were able to teach with more subtly: this is definitely not a car, it is very lizard like and a little snake like.

Re:The Unreasonable Effectiveness of Recurrent Neural Networks by kimbly on Tuesday May 26, 2015 @09:12PM (#9608705)
Do you have a reference?
Re:The Unreasonable Effectiveness of Recurrent Neural Networks by sinwave on Friday May 29, 2015 @01:56AM (#9622927)
Geoffrey Hinton gave results to this effect in a talk about "Dark Knowledge" [1]. Haven't seen any of these results published, though. I think he mentions something in the talk about NIPS rejecting the paper.

[1] - https://www.youtube.com/watch?v=EK61htlw8hY