And when I say peer review, perhaps I meant to say peer reproduced (which wasn't super obvious in the initial comment).
Re:New human-like species discovered
by yawaramin on Thursday September 10, 2015 @09:07PM
(#10201474)
Good point; after all, how can a review be serious if the reviewer(s) don't even try to reproduce the results?
A peer review can identify a very large class of problems with research without actually replicating the experiment.
For example, problems with methodology and sampling, overlooked prior research, bad analysis of results, straight-up non-sequiturs and conclusions drawn from insufficient data.
All of these mean peer review is valuable without reproducing results.
For an extreme example: if somebody publishes a "Cheese found on moon" paper, do you expect the reviewer to mount an expedition to the moon to check?
Re:New human-like species discovered
by yawaramin on Thursday September 10, 2015 @09:45PM
(#10201566)
Those are all excellent points, but even if you don't find any significant problems with the methodology or the math, or any of the other factors, that still doesn't make a complete review--not without replication.
For the 'Cheese found on moon' paper, I would expect the reviewer to clearly state that the results could not be replicated before mentioning anything else. Let readers draw informed conclusions based on a complete picture of the result, including whether or not it was a one-off.
With rampant forging of data and results, you better be skeptical of peer-reviewed results too. Reproducibility is an important, oft-overlooked, and under-appreciated part of science.