News for hackers. Stuff that matters.
Re:Redefine statistical significance by jimmar on Thursday September 07, 2017 @01:50PM (#15194138)
Not a huge fan of this idea. For example, people who analyze twitter data can get very small p-values because they analyze millions of tweets even though the effects they find are very small. See https://papers.ssrn.com/sol3/papers.cfm?abstract_id=1336700
Re:Redefine statistical significance by moultano on Thursday September 07, 2017 @02:12PM (#15194347)
I'd rather hear about small things that are true than large things that are false.
Re:Redefine statistical significance by cropsieboss on Thursday September 07, 2017 @02:45PM (#15194648)
The things is that these small things might just be noise from some confounding factors.

http://jaoa.org/article.aspx?articleid=2517494

For example, here sample size is huge, USA population gets significant increased risk, while EU population does not. Mixing the two together would result in a smaller but still significant increased risk.

Given the size, it's quite clear that USA population has many other confounding factors that cannot be eliminated by mathematics alone (there is no control).

Re:Redefine statistical significance by nonbel on Thursday September 07, 2017 @10:38PM (#15197513)
Statistical significance doesn't mean an effect is "true", or "real". So getting rid if it shouldn't cause you any issues.
Re:Redefine statistical significance by stewbrew on Friday September 08, 2017 @03:29AM (#15198469)
The problem here is that you most likely hear about false positives.
Re:Redefine statistical significance by amelius on Thursday September 07, 2017 @02:11PM (#15194336)
I don't see the problem as long as you clearly separate significance and effect-size.
Re:Redefine statistical significance by seanwilson on Thursday September 07, 2017 @05:30PM (#15195930)
This bothers me a lot of media reporting. The headline will be something like "X is good/bad for you" with a tiny effect size but the way it's reported makes you assume it's a large effect size. Usually the effect size won't be discussed in any depth or at all, they just want to sum it up in black and white. If the effect size is tiny it's probably just noise that would disappear in a higher quality future study.
Re:Redefine statistical significance by mhermher on Thursday September 07, 2017 @02:47PM (#15194678)
This is why I think better multiple-test-correction methods are more important and would lead better to a desirable outcome than just lowering alpha.
Re:Redefine statistical significance by stewbrew on Friday September 08, 2017 @03:32AM (#15198485)
Multiple-test correction usually implies lowering alpha, doesn't it?
Re:Redefine statistical significance by mhermher on Friday September 08, 2017 @03:51PM (#15203412)
For any given test yes, but it usually tries to maintain your overall alpha at the same level.