News for hackers. Stuff that matters.
Re:Netflix Prize Update: New Team in First Place (deadline t... by gtzi on Sunday July 26, 2009 @03:58AM (#724184)
The Ensemble is actually a meta-team of most of the teams that didn't participate in BPC, and I'm hearing that BPC had lately tried to get some other teams to join, so they were kinda expecting that (but I don't think they will be able to fire back). Bottom line, I think that the most massive open data set out there (next to a competition) has really helped the research to move on, but I don't see the point in 5 or 10 teams joining forces to improve the results for a tiny percentage and win the prize. On the other hand, maybe this is actually the right thing, for all these teams that have struggled for two years to be in the top-15 to receive the prize...
Re:Netflix Prize Update: New Team in First Place (deadline t... by rudyfink on Sunday July 26, 2009 @12:43PM (#724550)
> I don't see the point in 5 or 10 teams joining forces to improve the results for a tiny percentage and win the prize.

I think the point is who gets the million dollars. It makes sense that people would group up, probably into as few additional participants as it would take to ensure winning. If you and I were in the vicinity of being able to win independently and we knew we would lose otherwise, I think we have every incentive to team up.

Re:Netflix Prize Update: New Team in First Place (deadline t... by hvs on Sunday July 26, 2009 @12:49PM (#724563)
BPC is a meta-team as well. It's made up of four other teams. Some members of BPC won the Progress Prizes in previous years.

I can completely understand joining forces to win the prize. $1,000,000 isn't chump change, and a smaller percentage of it is better than none.

Re:Netflix Prize Update: New Team in First Place (deadline t... by IgorCarron on Sunday July 26, 2009 @04:17AM (#724193)
Shouldn't a new result above 10 pct reset the clock with a new one month period ?
Re:Netflix Prize Update: New Team in First Place (deadline t... by slackenerny on Sunday July 26, 2009 @09:27AM (#724383)
From the rules: http://www.netflixprize.com/rules

When the RMSE of a submitted prediction set on the quiz subset improves beyond the qualifying RMSE an electronic announcement will inform all registered Participants that they have thirty (30) days to submit additional candidate prediction sets to be considered for judging. At the end of this period, qualifying submissions will be judged (see Judging below) in order of the largest improvement over the qualifying RMSE on the test subset.

Re:Netflix Prize Update: New Team in First Place (deadline t... by hvs on Sunday July 26, 2009 @12:46PM (#724559)
Maybe it should, maybe it shouldn't, but Netflix defined it as 30 days from the first submission over a 10% improvement. I'm sure there's some reason why they did it that way, but I don't know what it is.
Re:Netflix Prize Update: New Team in First Place (deadline t... by markbao on Sunday July 26, 2009 @03:32AM (#724170)
One thing's for sure: BellKor's Pragmatic Chaos is going to get shitfaced tonight.
Re:Netflix Prize Update: New Team in First Place (deadline t... by Raphael on Sunday July 26, 2009 @03:37AM (#724173)
Trying to hit the Ballmer Peak.
Re:Netflix Prize Update: New Team in First Place (deadline t... by markbao on Sunday July 26, 2009 @03:39AM (#724174)
BK/PC submitted their last RMSE 2009-07-25 15:53:34, three hours before The Ensemble. If they can hit the Ballmer Peak or otherwise beat the Ensemble RMSE before tomorrow morning, it should be very interesting.
Re:Netflix Prize Update: New Team in First Place (deadline t... by Herring on Sunday July 26, 2009 @05:06AM (#724221)
Aren't you only allowed to submit once every 24 hours or something?
Re:Netflix Prize Update: New Team in First Place (deadline t... by hvs on Sunday July 26, 2009 @12:44PM (#724554)
Yes, you have to wait 24 hours from your last submission.
Re:Netflix Prize Update: New Team in First Place (deadline t... by Herring on Sunday July 26, 2009 @03:54AM (#724181)
Anyone know why blending seems to work so well?
Re:Netflix Prize Update: New Team in First Place (deadline t... by lliiffee on Sunday July 26, 2009 @12:51PM (#724566)
I believe it is a consequence of A) netflix evaluating the algorithms in terms of the squared error and B) different classification algorithms making different types of mistakes.

Take a simple example, with two movies (A and B), and two algorithms (X and Y). Say Alg. X rates movie A perfectly, but is off by 2 for movie B. Meanwhile, Alg. Y rates movie B perfectly, but is off by 2 for movie A.

The squared errors for the different algorithms will be:

Alg X: 4 (=0^2 + 2^2)

Alg Y: 4 (=2^2 + 0^2)

Mean of Alg X and Alg Y: 2 (=1^2 + 1^2)

Re:Netflix Prize Update: New Team in First Place (deadline t... by mattj on Sunday July 26, 2009 @04:03AM (#724185)
It's a standard ml technique that's been shown to work well, although no ones shown a proof of this. This is related to the observation "more data beats better models." This does seem to have it's limits - at some point you do need better features before you can expect your data to pay off.
Re:Netflix Prize Update: New Team in First Place (deadline t... by gaika on Sunday July 26, 2009 @04:04AM (#724187)
Only because each individual algorithm is slightly overfit. Blending it all together keeps the good predictive part and reduces the overfit noise.
Re:Netflix Prize Update: New Team in First Place (deadline t... by hundredwatt on Sunday July 26, 2009 @09:45AM (#724393)
It seems like just a consequence of basic statistics (central limit theorem?). Any algorithm you create has some random bias. By blending, you take several algorithms with random bias and average them together, reducing the overall bias (assuming the algorithms converge toward the correct result).
Re:Netflix Prize Update: New Team in First Place (deadline t... by eru on Sunday July 26, 2009 @11:03AM (#724453)
That's as far from the central limit theorem as you can go..
Re:Netflix Prize Update: New Team in First Place (deadline t... by trapper on Sunday July 26, 2009 @02:12AM (#724121)
Wow, the second team will be gutted!
Re:Netflix Prize Update: New Team in First Place (deadline t... by harknesslabs on Sunday July 26, 2009 @03:43AM (#724176)
If BPC didnt expect this to happen they deserve to lose. They should have expected a last minute submission and have one for themselves also.
Re:Netflix Prize Update: New Team in First Place (deadline t... by hvs on Sunday July 26, 2009 @12:28PM (#724534)
Reducing the RMSE isn't quite that simple. Teams have been working for years to improve the algorithm by 10%. You can't just "boost it up" in order to win. It takes serious work. BPC is still working to improve their algorithm like everyone else but questioning whether they "expected this to happen" is ridiculous. None of the top teams "deserve to lose", they've all done great work and deserve a round of applause, regardless if they are the ultimate winner.
Re:Netflix Prize Update: New Team in First Place (deadline t... by colins_pride on Sunday July 26, 2009 @08:46AM (#724349)
Yeah, the timestamp right there next to BPC is 07-25
Re:Netflix Prize Update: New Team in First Place (deadline t... by gojomo on Sunday July 26, 2009 @02:18AM (#724124)
Is there any rolling-extension each time a new top submission arrives -- like in some auctions -- or is the deadline firm at X days after the first +10% entry?
Re:Netflix Prize Update: New Team in First Place (deadline t... by elq on Sunday July 26, 2009 @02:27AM (#724131)
no. it's over tomorrow morning at ~11:30am PST... there are a few complications that could cause an extension, but they're not likely.