News for hackers. Stuff that matters.
Re:The Absurdity of Peer Review: What the Pandemic Revealed by makomk on Sunday June 06, 2021 @05:51PM (#27416686)
One of the worst hydroxychloroquine studies - the now-retracted one in the Lancet claiming that patients receiving the drug were dying at a higher rate, which appears to have used completely fake data and where there was no possible way they could've got the data they purported to have used - was in fact peer reviewed. Peer review really doesn't do as much as people seem to think it does.
Re:The Absurdity of Peer Review: What the Pandemic Revealed by tinus_hn on Sunday June 06, 2021 @07:38PM (#27417410)
Peer review can’t catch fake data. Peer review can look at if an experiment is repeatable and repeating the experiment can show fake data.
Re:The Absurdity of Peer Review: What the Pandemic Revealed by beagle3 on Monday June 07, 2021 @04:18AM (#27420264)
Peer review definitely could and should have caught the fake data in this case - there were too many implausible things about this paper that one could verify with a google search and see cannot be right.

In fact, it took people about one day from publication to start pointing them out with proof - and these people were (supposedly) less qualified than the peer reviewers.

It is true that, with properly faked data, a reviewer cannot be expected to catch it. Nor can p-hacking generally be discovered from an incomplete description.

But quite a bit of faked and even p-hacked data is less-than-competently done in a way that a reviewer can and should be able to catch;

And observational data or meta-study are much easier to verify, since no experiment was done by the researcher either.

Re:The Absurdity of Peer Review: What the Pandemic Revealed by xg15 on Sunday June 06, 2021 @05:59PM (#27416739)
So then what? Even if peer review does not solve the problem of bad research completely, it certainly mitigates it better than not doing anything.

How would you solve this problem?

Re:The Absurdity of Peer Review: What the Pandemic Revealed by User23 on Sunday June 06, 2021 @06:41PM (#27417016)
> Even if peer review does not solve the problem of bad research completely, it certainly mitigates it better than not doing anything.

This isn't self-evident. In fact, it's conceivable that peer review could actually lower the quality of published research by delaying paradigm shifting work from seeing the light of day.

Re:The Absurdity of Peer Review: What the Pandemic Revealed by TigeriusKirk on Sunday June 06, 2021 @06:29PM (#27416953)
By attributing less weight to the imprimatur of peer review.

It's a sign that something might well hold up, but it's not definitive. I think people know this about their own field, and then immediately forget it when dealing with another field.

Re:The Absurdity of Peer Review: What the Pandemic Revealed by IncRnd on Sunday June 06, 2021 @06:44PM (#27417030)
How? If someone is going to rely upon the conclusions of a paper, simply read that paper and evaluate the claims. i.e. stop the sole use of the titles of papers and their cliffs notes to support a point.
Re:The Absurdity of Peer Review: What the Pandemic Revealed by bazooka_penguin on Monday June 07, 2021 @02:30AM (#27419640)
I think the problem people are getting is that "peer reviewed" has become a badge of expertise or authority in itself, inflating people's trust in it and weakening skepticism.
Re:The Absurdity of Peer Review: What the Pandemic Revealed by AwaAwa on Sunday June 06, 2021 @06:13PM (#27416847)
The problem is that "peers" of a like mind are easily found, when churning papers is rewarded. Quantity beats Quality in today's world.