News for hackers. Stuff that matters.
Re:Royal Society cautions against censorship of scientific m... by zrm on Monday January 31, 2022 @02:37AM (#30145493)
> Perhaps depositing a VM with everything ready to run.

The danger then is that the VM has so much undocumented complexity that if anything goes wrong, or goes "well" when it shouldn't, no one can explain why. Which also reintroduces a vector to hide nasty tricks.

Re:Royal Society cautions against censorship of scientific m... by hirako2000 on Monday January 31, 2022 @05:42AM (#30146618)
this. the point of sharing reproducible steps and not the experiment itself is that it can be fully reproduced independently. not just independently verify that the result show what the paper claims.
Re:Royal Society cautions against censorship of scientific m... by ungamedplayer on Monday January 31, 2022 @03:59AM (#30145982)
But you can at least figure that out. If U don't have it all results are suspect.
Re:Royal Society cautions against censorship of scientific m... by kergonath on Monday January 31, 2022 @07:48AM (#30147347)
Results that haven't been independently replicated are suspect. There are just too many factors that can lead an experiment to give some results that are not transferrable or not relevant.

The worst aspect of this is the lack of will or funding to replicate, replicate, and replicate again all significant results that get published. Post-processed data can be altered, but a TB of raw data is meaningless as well if it hasn't been produced properly, has been obfuscated, or is weirdly formatted.

Data availability is a red herring for the vast majority of the science being made right now (almost everything that does not depend on a multi-millions dollars experiment). If data availability is an end in itself, we would just have moved the goalposts and have a data quality problem instead of a reproducibility problem.