News for hackers. Stuff that matters.
Re:Replace Peer Review with “Peer Replication” (2021) by dongping on Sunday August 06, 2023 @11:31AM (#37023009)
While it is a lot of work, I tend to think that one can then always publish preprints if they can't wait for the replication. I don't understand why a published paper should count as an achievement (against tenure or funding) at all before the work is replicated. The current model just creates perverse incentives to encourage lying, P-hacking, and cherry-picking. This would at least work for fields like machine learning.

This is, of course, a naive proposal without too much thought into it. But I was wondering what I would have missed here.

Re:Replace Peer Review with “Peer Replication” (2021) by i_no_can_eat on Sunday August 06, 2023 @12:49PM (#37023887)
and in this proposal, who will be tasked with replicating the work?
Re:Replace Peer Review with “Peer Replication” (2021) by dongping on Sunday August 06, 2023 @03:14PM (#37025499)
In some fields, replication is already the prerequisite to benchmark the SoTA. So the incentives boil down to publishing them along with negative results. Or as some have suggested, make it mandatory for PHD candidates to replicate.

Though, it seems that it is possible to game the system, by creating positive/negative replication intentionally, to collude with/harm the author.