News for hackers. Stuff that matters.
Re:Scientists rise up against statistical significance by whatshisface on Wednesday March 20, 2019 @04:18PM (#19446401)
>Eg., your sight is defective, ie., in many cases: your sample is biased, not big enough, etc.

You can't expect a statistical test to comb through your experimental procedure to check whether or not you did it right. (Least of all, to comb through your personal medical history. ;) )

>Any observation is consistent with an infinite number of models.

I'm afraid that slips below the "get out of bed in the morning" pragmatism bar. Although it's a provocative idea, concluding that conclusions are impossible because you could be making a mistake or hallucinating would lead to, literally, not getting out of bed in the morning, if you took it seriously.

At the end of the day you just have to hope that the human beings reading your paper will be able to tell whether or not you did the experiment right. Any statistical test just walls off one way (or a few ways) in which you could fool yourself.

>Surveys of hundreds of articles have found that statistically non-significant results are interpreted as indicating ‘no difference’ or ‘no effect’ in around half (see ‘Wrong interpretations’ and Supplementary Information).

What we really want to avoid here is requiring every paper to have a twelve paragraph Sophist disclaimer about how knowledge is impossible. ;) Just because the text doesn't say "oh and by the way there could be gnomes in the LHC that are fooling us to entertain themselves," doesn't mean that the author wouldn't begrudgingly acknowledge it as an epistemic risk.

Re:Scientists rise up against statistical significance by mjburgess on Wednesday March 20, 2019 @04:28PM (#19446493)
Well your "pragmatism bar" is higher that all future science.

It isnt a trivial sceptical point.

Does CO2 cause climate change, or climate change cause C02, or are they both caused by a hidden factor (one of many) or do they co-vary accidentally (ie., no joint cause)?

No "statistical significance" of any kind can decide between them, yet this is how it is used. There is no way to do so with statistics; it requires repeated experimentation and refutation of all plausible causal models until the one remains stands as "the most plausible".

"Statistics" is being used in place of the scientific method which it cannot replace. Covariation is not causation: the covarying of any number of variables is consistent with any number of causal models. No "test" decides.

Re:Scientists rise up against statistical significance by whatshisface on Wednesday March 20, 2019 @04:33PM (#19446545)
The evidence for anthropogenic climate change involves a lot more than correlation, so maybe that isn't the best example. In any case, even if everyone on Earth is misinterpreting p-values, that's not a problem with p-values.
Re:Scientists rise up against statistical significance by mch82 on Wednesday March 20, 2019 @05:07PM (#19446935)
The point of statistics isn’t to attain absolute certainty or for the statistical test to make a decision.

The point is for people to make a decision using probability as one available decision aid, and then deciding whether to place a bet.

Re:Scientists rise up against statistical significance by pmoriarty on Wednesday March 20, 2019 @04:28PM (#19446495)
"concluding that conclusions are impossible because you could be making a mistake or hallucinating would lead to, literally, not getting out of bed in the morning, if you took it seriously."

Not if you don't have free will, which is apparently the conclusion that a lot of scientists seem to be coming to recently.

Re:Scientists rise up against statistical significance by whatshisface on Wednesday March 20, 2019 @04:30PM (#19446517)
Haha, then allow me to rephrase. A brain whose molecules react that conclusions are impossible will no longer chemically get out of bed in the morning.
Re:Scientists rise up against statistical significance by pmoriarty on Wednesday March 20, 2019 @04:46PM (#19446673)
Why can't you act in ways that are at odds with your conclusions?

When I look around me I see people doing this every day.