Re:Recommender Systems: We're doing it (all) wrong
by xamat on Friday April 08, 2011 @10:22AM
(#2423909)
Hi @kenjackson, this is Xavier here. How do you propose to validate this on the Netflix dataset? It is clear that you cannot use RMSE to compare to other existing approaches, right? The way to go would be to propose a different success measure (i.e. ranking based) and measure how different algorithm perform. And then validate this on users to prove that optimizing RMSE is not as useful.
If you give me a few months, I might get there. But this is the reason I wrote a blog post and not a paper ;-)
Re:Recommender Systems: We're doing it (all) wrong
by kenjackson on Friday April 08, 2011 @12:02PM
(#2424354)
If you give me a few months, I might get there. But this is the reason I wrote a blog post and not a paper ;-)
Fair enough. I'll keep my eyes peeled for a paper in a few months. :-)
Re:Recommender Systems: We're doing it (all) wrong
by frak_your_couch on Friday April 08, 2011 @11:21AM
(#2424153)
I just wanted to point out that the link on your blog to PureSVD points to the wikipedia page for Discounted Cumulative Gain.
Re:Recommender Systems: We're doing it (all) wrong
by xamat on Friday April 08, 2011 @04:21PM
(#2425450)
OOps! Thanks... just fixed it.
Re:Recommender Systems: We're doing it (all) wrong
by frak_your_couch on Friday April 08, 2011 @08:26PM
(#2426097)
Hey, sorry, one more time. The updated link is also wrong, it's relative to your blog rather than absolute i.e. http://technocalifornia.blogspot.com/2011/04/research.yahoo.... as opposed to http://research.yahoo.com/files/recsys2010_submission_150.pd...
There is a great data set to test this theory... Netflix. This article shouldn't end by just solicitng opinions, but with his results in the Netflix data set.