News for hackers. Stuff that matters.
Re:Half the papers at NIPS would be rejected if the review p... by liquidise on Monday December 15, 2014 @06:35PM (#8755115)
I am more curious of the inverse: what if more that 22.5% of papers were at acceptable quality levels? Wouldn't that leave each committee to pick and choose, thus artificially inflating their disagreements?
Re:Half the papers at NIPS would be rejected if the review p... by wsxcde on Monday December 15, 2014 @07:03PM (#8755238)
Yes, and I think that is essentially why we're seeing these disagreements.

I've heard from lots of professors that a good conference gets a lot of "very-good-but-not-great" submissions and the job of the program committee is to pick the best among these. I wouldn't be surprised at all if minor personal preferences (which from the outside look rather random) ended up having a big say in the fate of a particular paper. Maybe some reviewers are more forgiving of poorly-written but technically strong papers, maybe some reviews consider certain fields "dead" and so are biased against them, reviewers tend to wildly different standards on how extensive an experimental analysis should be to be acceptable, ...