News for hackers. Stuff that matters.
Re:Three myths about scientific peer review by cperciva on Friday January 09, 2009 @07:41PM (#427391)
While I agree with the author's main points about peer review, I think it's a bit of a stretch to conclude from those that peer review is useless.

As a peer reviewer, I've sent papers back with comments along the lines of "this work seems quite similar to the published work of X in the following papers (which the author doesn't cite): ... I wonder if he if the author is aware of this earlier work", and subsequently heard nothing more (presumably because, upon looking at said papers, the author came to the conclusion that he had not in fact discovered anything new). On the opposite side of the coin, one of my papers benefited from very similar comments, in that having related work brought to my attention allowed me to revise my paper to better describe how it fit with the rest of the field.

I think it's in cases like this -- where peer reviewers point out things (whether other research, or possible errors) of which the author(s) do not seem to be aware -- that peer review really shines. If reviewers respond merely with a 'yea' or 'nay' they're not doing their job properly; their job is to review, not to judge.

Re:Three myths about scientific peer review by yummyfajitas on Saturday January 10, 2009 @09:12AM (#427935)
The quality of peer review varies with the journal. For instance, Journal of Computational Physics tends to give out helpful reviews. From JCP, I usually get back a list of helpful suggestions and weak spots in the paper. Even the incorrect suggestions/criticisms tend to reveal ambiguity in my writing more than anything else.

PRL, not so much. In my experience, at least one referee doesn't even finish reading the paper. It's not just me, this happened to a friend of mine last week:

Referee: "The authors do not demonstrate that their equations apply to any physical example."

Response: "See page 3 paragraph 4."

Re:Three myths about scientific peer review by neilc on Friday January 09, 2009 @09:42PM (#427504)
I think it's a bit of a stretch to conclude from those that peer review is useless.

I don't think that's what the author concludes. But otherwise, I agree with you.