News for hackers. Stuff that matters.
Re:Potential fabrication in research threatens the amyloid t... by vlovich123 on Friday July 22, 2022 @02:10AM (#32189062)
I wonder what it would look like if you’re not allowed to cite papers in the field unless there’s been 5 independent labs that have managed to reproduce the result and all the labs that reproduce get to share in the credit (eg if there’s a Nobel prize). Would that change the incentive structure and create better outcomes? Science isn’t science until you have a repeatable process in place where observations match outcomes and the underlying theory. Someone proposing a bold new theory without the ability to test it usually gets significant credit even if posthumously for being a bold visionary which seems to have been sufficient throughout time as a motivator for the really foundational scientific victories.
Re:Potential fabrication in research threatens the amyloid t... by toxik on Friday July 22, 2022 @09:16AM (#32191565)
The effort required to reproduce a result, especially an exotic one, is often prohibitive. The reward for successful reproduction is near zero, and if you fail, it’s often absolutely zero - you are not going to be able to contest a result by a failure. The original authors can simply say that your reproduction was flawed, if they need to answer at all.

A journal that requires five confirmations to cite a work is a journal that will fast have zero submissions.

The main reason people attempt reproduction is to continue that line of research, and that often starts with reproduction plus a slight tweak. This is why you find a lot of papers like this. This is also how irreproducible results become sort of known about but not challenged in many fields.

Re:Potential fabrication in research threatens the amyloid t... by ebiester on Friday July 22, 2022 @09:19AM (#32191605)
So what does the incentive structure look like? Do you get tenure for running a lab that replicates? Are your postdocs in line for tenure track positions? How is it funded: do you put the secondary replications into the original grant? What happens if you can't find X number of labs to run the test?

It's an interesting idea, but it all comes down to how the incentive structures happen.

Re:Potential fabrication in research threatens the amyloid t... by vlovich123 on Saturday July 23, 2022 @12:26AM (#32200351)
Yes. You get tenure track debunking things that have mind share in the community and adding support for theories that become foundational.