Re:Netflix Prize Update: New Team in First Place (deadline t...
by Herring on Sunday July 26, 2009 @03:54AM
(#724181)
Anyone know why blending seems to work so well?
[ Parent
| on HN ]
Re:Netflix Prize Update: New Team in First Place (deadline t...
by lliiffee on Sunday July 26, 2009 @12:51PM
(#724566)
I believe it is a consequence of A) netflix evaluating the algorithms in terms of the squared error and B) different classification algorithms making different types of mistakes.
Take a simple example, with two movies (A and B), and two algorithms (X and Y). Say Alg. X rates movie A perfectly, but is off by 2 for movie B. Meanwhile, Alg. Y rates movie B perfectly, but is off by 2 for movie A.
The squared errors for the different algorithms will be:
Alg X: 4 (=0^2 + 2^2)
Alg Y: 4 (=2^2 + 0^2)
Mean of Alg X and Alg Y: 2 (=1^2 + 1^2)
Re:Netflix Prize Update: New Team in First Place (deadline t...
by mattj on Sunday July 26, 2009 @04:03AM
(#724185)
It's a standard ml technique that's been shown to work well, although no ones shown a proof of this. This is related to the observation "more data beats better models." This does seem to have it's limits - at some point you do need better features before you can expect your data to pay off.
Re:Netflix Prize Update: New Team in First Place (deadline t...
by gaika on Sunday July 26, 2009 @04:04AM
(#724187)
Only because each individual algorithm is slightly overfit. Blending it all together keeps the good predictive part and reduces the overfit noise.
Re:Netflix Prize Update: New Team in First Place (deadline t...
by hundredwatt on Sunday July 26, 2009 @09:45AM
(#724393)
It seems like just a consequence of basic statistics (central limit theorem?). Any algorithm you create has some random bias. By blending, you take several algorithms with random bias and average them together, reducing the overall bias (assuming the algorithms converge toward the correct result).