News for hackers. Stuff that matters.

A comment on 1 in 4 Statisticians Say They Were Asked to Commit Scientific Fraud. Parent comment

Re:1 in 4 Statisticians Say They Were Asked to Commit Scient... by ascar on Saturday November 03, 2018 @08:45AM (#18369934)
This can't be upvoted enough. Makes me thinking that using published source code and reproducing/validating results is completely orthogonal. Maybe it's a good thing the source code for science gets not published.
[ Parent | on HN ]
Re:1 in 4 Statisticians Say They Were Asked to Commit Scient... by BeetleB on Saturday November 03, 2018 @02:20PM (#18371653)
I like the concept of rewriting all the code by an unbiased third party to see if they can reproduce the results, but in practice what this leads to is:

1. People will not bother. It took a lot of minds to come up with the software used (in my friend's case, several PhD's amount of work). No one is going to invest that much effort to invent their own software libraries to get it to work.

2. Even when you do write your own version of the software, there are a lot of subtleties involved in, say, computational physics. Choices you make (inadvertently) affect the convergence and accuracy. My producing a software that gives different results could mean I had a bug. It could mean they did. It could mean we both did. Until both our codes are in the open, no one can know.

It is very unlikely that you'll have a case of one group's software giving one result and everyone else's giving another. More like everyone else giving different results.

Case in point:

https://physicstoday.scitation.org/do/10.1063/PT.6.1.2018082...

HN discussion:

https://news.ycombinator.com/item?id=17819420

[ Parent | on HN ]