Re:Why Most Published Research Findings Are False (2005)
by gus_massa on Sunday September 30, 2018 @03:51PM
(#18108008)
From your profile, you are working in Deep Learning. In other fields it's much more difficult to be sure that the replication is accurate.
Do you have the correct variety of rats? Are they receiving the same kind of food? Does they get the same illumination during the day? ... Theoretically al the details should be clear from the published paper, but most of the times the paper is full of "underspecified"[0] parts.
Also, in many fields even if the paper is only about calculations in a computer, the programs are not published (or are a mess (or an unpublished mess)) and the data are not published (or are a mess (or an unpublished mess)). So
it's more difficult to even make a direct copy of the results of the paper.
Also, there is a lot of informal replication, essentially what you do but without the final publication step. Just replicate somewhat similar to the original paper, but then publish a version with some extension or tweak.
Do you have the correct variety of rats? Are they receiving the same kind of food? Does they get the same illumination during the day? ... Theoretically al the details should be clear from the published paper, but most of the times the paper is full of "underspecified"[0] parts.
Also, in many fields even if the paper is only about calculations in a computer, the programs are not published (or are a mess (or an unpublished mess)) and the data are not published (or are a mess (or an unpublished mess)). So it's more difficult to even make a direct copy of the results of the paper.
Also, there is a lot of informal replication, essentially what you do but without the final publication step. Just replicate somewhat similar to the original paper, but then publish a version with some extension or tweak.
[0] aka "missing"