News for hackers. Stuff that matters.

A comment on Ask HN: Should I publish my research code?. Parent comment

Re:Ask HN: Should I publish my research code? by allenofthehills on Friday January 14, 2022 @11:22AM (#29936412)
This is a common misconception, but you are actually talking about "replicability" which is "writing and then running new software based on the description of a computational model or method provided in the original publication, and obtaining results that are similar enough" [1]. Reproducibility instead refers to running the same code on the same data to get the same results [2].

[1] Rougier et al., "Sustainable computational science: the ReScience initiative" https://arxiv.org/abs/1707.04393

[2] Plesser, "Reproducibility vs. Replicability: A Brief History of a Confused Terminology" https://doi.org/10.3389/fninf.2017.00076

[ Parent | on HN ]
Re:Ask HN: Should I publish my research code? by jpeloquin on Friday January 14, 2022 @05:18PM (#29941281)
It's worth noting that the two provided references disagree on their definitions of reproducibility. Plesser quotes ACM as reproducibility being tested by a "Different team, different experimental setup", consistent with use in the GP's comment, but ultimately seems to favor ignoring "replicability" and instead using "reproducibility" in combination with various adjectives. The common understanding in the physical sciences is also that merely re-running the same code is not a test of reproducibility.
[ Parent | on HN ]