News for hackers. Stuff that matters.
Re:Scientists rise up against statistical significance by amluto on Wednesday March 20, 2019 @06:16PM (#19447553)
> Raising threshold will not just reduce probability of false positive result, but also will raise probability of false negative.

Not really. You raise the probability of an inconclusive result when you could otherwise have gotten a positive. If you interpret p > threshold as “null hypothesis is true”, then you are doing the statistics wrong.

In most cases, I think a better model would be to extract an effect size such that an effect larger than the size is ruled out by the study to some degree of confidence. Currently, I read about studies that conclude that “such-and-such had no significant effect detected by this study.” Concretely, this looks like “vaccines had no significant effect on autism risk.” This may be accurate, but it’s lousy. How about “vaccines caused no more than an 0.01% increase in autism, and a bigger study could have set an even tighter limit.”

Physicists regularly do this. For example, we know that the universe has no overall curvature to a very good approximation.

Re:Scientists rise up against statistical significance by Fomite on Thursday March 21, 2019 @03:13AM (#19449775)
We already see this happening in the clinical literature though. Effect estimates that show a positive clinical outcome, but have confidence intervals that barely brush the null are described as having "no impact".