News for hackers. Stuff that matters.
Re:Let’s make peer review scientific by Fomite on Wednesday July 06, 2016 @04:08PM (#12045526)
I really dislike the idea that "Your code runs on your data" is a form of replication.
Re:Let’s make peer review scientific by apathy on Wednesday July 06, 2016 @06:24PM (#12046285)
It's a minimum standard and one that many published works cannot meet. Universal enforcement of this minimum would itself be a big step forward.
Re:Let’s make peer review scientific by Fomite on Wednesday July 06, 2016 @06:39PM (#12046351)
It is a minimum standard, and a useful one. It mostly annoys me when certain groups of advocates seem to treat it as an end in and of itself, and evidence that something is "repeatable" and will end the problems in obtaining scientific evidence.

I come from a field where something isn't a reproduced until it's also found in an entirely different study on a different population.

Re:Let’s make peer review scientific by apathy on Wednesday July 06, 2016 @07:30PM (#12046564)
Oh, same here. Two of my current manuscripts have now been replicated in 3-5 independent clinical cohorts. The thing is, if the field won't even recognize that "ability to produce the same outputs given your inputs" is critical, good fucking luck with things like "actually replicates in a separate population".

Neuroscience is particularly horrendous about this.