What happens when you try to publish a failure to replicate in 2015/2016

Re:What happens when you try to publish a failure to replica... by faux_intellect on Wednesday June 29, 2016 @01:53AM (#11999759)

At this point, I think Google Scholar should step in and just put a replications section beside every scientific publication. People should be able to quickly and easily know how many times a study has been attempted to replicate and, of those attempted, how many times it has actually successfully replicated.

It's unfortunate that replications aren't taken more seriously these days, but it also doesn't help that, when there are actual replications, you have to scour the internet for them rather than having them readily available to you.

[ Parent | on HN ]

Re:What happens when you try to publish a failure to replica... by kriro on Wednesday June 29, 2016 @02:01AM (#11999787)

I think that's a pretty great idea actually. If anyone here has the reach to get this on their table please do reach out. It would be even greater if they'd use some replication metric in their ranking algorithm for the papers.

The danger however is that this would just lead to only me-too studies being accepted and failed replications still being rejected.

I'd also love if they'd add (possibly Google hosted) repositories where the data/scripts etc. that belong to the paper can be uploaded and archived for all eternity.

[ Parent | on HN ]

Re:What happens when you try to publish a failure to replica... by mwambua on Wednesday June 29, 2016 @05:11AM (#12000380)

I saw something like this a while ago... Codalab (http://codalab.org/). It's run by Percy Liang (https://cs.stanford.edu/~pliang/). It lets you create executable worksheets to go with your papers; and people can validate both the code and data used in your experiments.

[ Parent | on HN ]

Re:What happens when you try to publish a failure to replica... by chronic81 on Wednesday June 29, 2016 @06:35AM (#12000681)

http://gitxiv.com is far superior than Codalab. Source: I go to Stanford.

[ Parent | on HN ]

Re:What happens when you try to publish a failure to replica... by kriro on Thursday June 30, 2016 @02:05AM (#12006879)

Looks pretty cool indeed, bookmarked. However I believe it is focused on Computer Science. Nothing wrong with that but I think they key is convincing scientists that are not exposed to ideas like version control, open source (and I'd argue data sharing due to the rise of data mining) by default (or are less computer savvy in general).

[ Parent | on HN ]

Re:What happens when you try to publish a failure to replica... by mwambua on Wednesday June 29, 2016 @07:26AM (#12000881)

Interesting... It seems broken right now; but the github & arxiv integration sounds pretty cool, so I'll keep an eye on it.

[ Parent | on HN ]

Re:What happens when you try to publish a failure to replica... by dnautics on Wednesday June 29, 2016 @07:21AM (#12000853)