News for hackers. Stuff that matters.
Re:Replace Peer Review with “Peer Replication” (2021) by matthewdgreen on Sunday August 06, 2023 @04:25PM (#37026263)
In the current system scientists allocate reproduction efforts to results that they intend to build on. So if you’ve claimed a breakthrough technique for levitating widgets — and I think this widget technique can be used to build spacecraft (or if I think your technique is wrong) — then I will allocate precious time and resources to reproducing your work. By contrast if I don’t think your work is significant and worth following up on, then I allocate my efforts somewhere else. The advantage is that more apparently-significant results (“might cure cancer”) tend to get a bigger slice of very limited resources, while dead-end or useless results (“might slightly reduce flatulence in cats”) don’t. This distributed entrepreneurial approach isn’t perfect, but it works better than central planning. By contrast you could adopt a Soviet-like approach where cat farts and cancer both share replication resources, but this seems like it would be bad for everyone (except the cats.)
Re:Replace Peer Review with “Peer Replication” (2021) by GravelRocks on Sunday August 06, 2023 @09:36PM (#37028960)
"Cat fart" research might also be incredibly expensive to replicate compared to the "might cure cancer" research. In that case it would effectively get a bigger slice of resources because we're treating all research the same!
Re:Replace Peer Review with “Peer Replication” (2021) by SkyBelow on Monday August 07, 2023 @09:30AM (#37034209)
>In the current system scientists allocate reproduction efforts to results that they intend to build on. So if you’ve claimed a breakthrough technique for levitating widgets — and I think this widget technique can be used to build spacecraft (or if I think your technique is wrong) — then I will allocate precious time and resources to reproducing your work.

Wouldn't this imply that worthwile results are already being replicated, so the primary cost has been paid and we just need to give some method to disseminate this work and then have it factor into the credibility of science? pre-print < peer reviewed < peer replicated, with the last step having internal rankings depending upon how much it has been replicated?

And it'll also show when someone is building on things but not replicating it, which I guess is an issue in some fields more than others.