Re:Understanding Bias in Peer Review
by humanrebar on Friday December 01, 2017 @06:55AM
(#15823307)
> How are you being a devil's advocate?
Popular opinion among scientists, engineers, etc. is that science stands (and should stand) on its own without needing support from appeals to authority, bandwagon effects, etc.
Okay, but how was kcorbitt being a devil's advocate with respect to this paper?
Re:Understanding Bias in Peer Review
by humanrebar on Friday December 01, 2017 @10:45AM
(#15824474)
He was being devil's advocate with respect to popular opinion, not this particular paper.
The paper was pretty clear that the popular opinion (among those with knowledge of the topic) is mixed, with advocates for both single blind and double blind, and that that vast majority of evaluations is done single blind - that is to say, aligned with the kcorbitt's position.
What do you mean by "devil's advocate"? Because I can't make sense of how to apply either my understanding of the term, nor the description at Wikipedia, to kcorbitt's comment.
"In conclusion, the heart of our findings is that single-blind reviewers make use of information about authors and institutions. Specifically, single-blind reviewers are more likely to bid on papers from top institutions and more likely to recommend for acceptance papers from famous authors or top institutions, compared with their double-blind counterparts."
"The primary ethical question is whether this behavior is acceptable. In one interpretation, single-blind reviewers make use of prior information that may allow them to make better overall judgments. As a consequence, however, it may be that other work is disadvantaged, in the sense that two contributions of roughly equal merit might be scored differently by single-blind reviewers, in favor of the one from a top school, while double-blind reviewers may not show this bias as strongly."
Or are you arguing that there is no ethical concern at all? The introduction seems to cover the main pro and con arguments, as part of the justification for the need for experimental evidence.