News for hackers. Stuff that matters.
Re: Is it time to up the statistical standard for scientific... by qwerta on Wednesday November 13, 2013 @08:07AM (#6724909)
My teachers always told me that main criteria in "science" is possibility to verify and reproduce. Study which relies on statistics, but no raw data or code is presented, is not scientific.

It does not take millions to check for basic mistakes. One person with computer and free afternoon is enough.

It is like heaving open-source, but without any source code.

Re: Is it time to up the statistical standard for scientific... by dllthomas on Wednesday November 13, 2013 @08:21AM (#6724970)
While I think availability of data and code is important, note that it does not accomplish many of the functions of replication of experiments. If two people implement the described algorithm and get the same results, we can be a lot more confident than if two people run the same implementation, because the same implementation is more likely to have the same bugs.
Re: Is it time to up the statistical standard for scientific... by qwerta on Wednesday November 13, 2013 @08:29AM (#6725016)
Code has to be reviewed and _VERIFIED_. How can an article pass peer review, if nobody even checked code for basic mistakes?
Re: Is it time to up the statistical standard for scientific... by dllthomas on Wednesday November 13, 2013 @10:47AM (#6725918)
Yes, precisely. Publishing code and data is necessary for peer review, not for replication. Both are necessary parts of science.
Re: Is it time to up the statistical standard for scientific... by SamuelMulder on Wednesday November 13, 2013 @09:19AM (#6725271)
By this criteria, there is very little science going on anywhere. We don't teach philosophy of science enough anymore. In most of the fields that I'm familiar with, reproducibility is very hard and rarely bothered about by either the original researcher or others in the field.
Re: Is it time to up the statistical standard for scientific... by dllthomas on Wednesday November 13, 2013 @11:22AM (#6726213)
That's true, and it's a problem.
Re: Is it time to up the statistical standard for scientific... by SamuelMulder on Wednesday November 13, 2013 @03:30PM (#6728240)
Agreed, but I'm not sure the solution is upping the statistical standards so much as educating researchers about what the statistical standards actually mean, how to use them, and how to construct a valid, repeatable experiment.
Re: Is it time to up the statistical standard for scientific... by dllthomas on Wednesday November 13, 2013 @09:48PM (#6730215)
And, importantly, providing more incentive to actually attempt replication.