The Future of Peer Review

Re:The Future of Peer Review by vladoh on Sunday February 05, 2012 @05:56PM (#3555574)

I think it is not possible to achieve the quality of a peer review by "crowd review" or "social review". While most of the people are able to rate a funny picture or an interesting article on Internet, the review of an academic paper requires much more expertise in the field and knowledge of the related work. Therefore I don't believe that a truly objective rating can be created. Also, making a peer review is time consuming because you have to really understand the paper in detail and think about if the contributions are big enough and well explained, if the experiments are done properly and if the comparisons are fare and so on. This is not an easy thing to achieve by just throwing a lot of people on it (the users of the Internet).

Furthermore, when a paper is published in a well recognized journal or conference one has a guarantee of the quality of the paper. For example when I see something published at CVPR (Conference on Computer Vision and Pattern Recognition) I know it is certainly worth reading. I don't want to imagine how much more difficult it will be to find the valuable papers among all the bad stuff that is being submitted for review and rejected.

I don't want to say that the peer review process is perfect, but if I have to choose between "peer review" and "crowd review" I choose the former.

[ Parent | on HN ]

Re:The Future of Peer Review by RichardPrice on Sunday February 05, 2012 @06:39PM (#3555670)

Academics have always based reading decisions on recommendations from colleagues. This trend has accelerated since email and the web were invented, as your colleagues can communicate with you through more channels, and more recommendations go back and forth. Social review always has been a big part of research discovery, and it's become more important with the advent of the web.

Search engines play a huge role in driving research discovery, and their ranking systems are fundamentally based on crowd review - calculations using links, as in the case of Google and Bing, or citations, as in the case of Google Scholar.

One might wonder in theory whether a search engine could do a good job of ranking academic papers, or whether recommendations from one's colleagues is a viable way of discovering papers. But when you look, in practice, into how academics operate day to day, those channels are the ones that academics increasingly use to discover research.

[ Parent | on HN ]