News for hackers. Stuff that matters.
Re:Machine learning works spectacularly well, but mathematic... by dasboth on Friday December 04, 2015 @11:15AM (#10677134)
I was just listening to that podcast yesterday and loved the way he presented this information. I know next to nothing about NNs but I could pretty much follow everything that he said (from an intuitive sense, anyway, if not a technical one).

This "magic" of converging networks reminds me of how ensemble methods, such as random forests, are effective but people aren't sure why. There's certainly (AFAIK) no theoretical grounds to say that a bunch of random decision trees should yield universally good results.

Re:Machine learning works spectacularly well, but mathematic... by mziel on Friday December 04, 2015 @01:07PM (#10677928)
There are. It's all about variance reduction. See the Breiman's paper on Bagging. There's nothing special about random forest though (apart from the fact that a decision tree is a good learner, because of the nonlinearities for example), you can you ensemble learning with any "basic" learner.
Re:Machine learning works spectacularly well, but mathematic... by dasboth on Friday December 04, 2015 @02:19PM (#10678417)
I read Breiman's random forest paper, but I'll give his others a read too. Does the variance reduction effect apply to any ensemble method then?