Re:Redefine statistical significance
by jimmar on Thursday September 07, 2017 @01:50PM
(#15194138)
Not a huge fan of this idea. For example, people who analyze twitter data can get very small p-values because they analyze millions of tweets even though the effects they find are very small. See https://papers.ssrn.com/sol3/papers.cfm?abstract_id=1336700
[ Parent
| on HN ]
Re:Redefine statistical significance
by moultano on Thursday September 07, 2017 @02:12PM
(#15194347)
I'd rather hear about small things that are true than large things that are false.
Re:Redefine statistical significance
by cropsieboss on Thursday September 07, 2017 @02:45PM
(#15194648)
The things is that these small things might just be noise from some confounding factors.
http://jaoa.org/article.aspx?articleid=2517494
For example, here sample size is huge, USA population gets significant increased risk, while EU population does not. Mixing the two together would result in a smaller but still significant increased risk.
Given the size, it's quite clear that USA population has many other confounding factors that cannot be eliminated by mathematics alone (there is no control).
Statistical significance doesn't mean an effect is "true", or "real". So getting rid if it shouldn't cause you any issues.
The problem here is that you most likely hear about false positives.
I don't see the problem as long as you clearly separate significance and effect-size.
Re:Redefine statistical significance
by seanwilson on Thursday September 07, 2017 @05:30PM
(#15195930)
This bothers me a lot of media reporting. The headline will be something like "X is good/bad for you" with a tiny effect size but the way it's reported makes you assume it's a large effect size. Usually the effect size won't be discussed in any depth or at all, they just want to sum it up in black and white. If the effect size is tiny it's probably just noise that would disappear in a higher quality future study.
Re:Redefine statistical significance
by mhermher on Thursday September 07, 2017 @02:47PM
(#15194678)
This is why I think better multiple-test-correction methods are more important and would lead better to a desirable outcome than just lowering alpha.
Multiple-test correction usually implies lowering alpha, doesn't it?