Re:Faster-than-light neutrino result may have been due to ba...
by jules on Wednesday February 22, 2012 @05:56PM
(#3622896)
Physics is far ahead of other disciplines in this regard. Choosing your statistical test after you gather the data, selectively removing "outliers" after you gather the data, non-blind interpretation of pictures by humans who have a stake in the outcome and only publishing statistically significant results are all par for the course in e.g. neuroscience.
[ Parent
| on HN ]
Re:Faster-than-light neutrino result may have been due to ba...
by gammarator on Wednesday February 22, 2012 @07:06PM
(#3623163)
This paper [1] points out that commonly-used measures of statistical significance are downright meaningless when additional degrees of freedom are hidden in the way you describe.
[1] http://people.psych.cornell.edu/~jec7/pcd%20pubs/simmonsetal...
Re:Faster-than-light neutrino result may have been due to ba...
by jules on Wednesday February 22, 2012 @10:48PM
(#3623701)
It's even worse than that paper describes and this is something that every statistics 101 class worth its salt points out: if you are allowed to choose a statistical test after you've gathered your data you can prove any conclusion you want with arbitrarily high confidence. Note that the paper does not list choosing a test before you gather the data as a requirement. The only way to do meaningful statistics is the way splat describes: describe exactly how you're going to analyze it before you gather the data, then send the paper to a journal which decides whether to publish it before the data has been gathered, and then complete the paper by actually doing the experiment and adding the data to the paper.