News for hackers. Stuff that matters.

A comment on Registered clinical trials make positive findings vanish. Parent comment

Re:Registered clinical trials make positive findings vanish by Alex3917 on Tuesday August 18, 2015 @06:02PM (#10082508)
> At the standard p-value cutoff of .05, even when there's absolutely no effect going on the probability of getting a spurious positive result when you do n comparisons is equal to 1 - (.95^n).

That's only the probability of getting a positive result by chance; the total probability of getting a positive result even if there is no effect is much higher.

[ Parent | on HN ]
Re:Registered clinical trials make positive findings vanish by rcthompson on Tuesday August 18, 2015 @07:27PM (#10082855)
"Probability of getting a positive result by chance" and "probability of getting a positive result when there is no effect" are the same thing. This is what the p-value measures. (The full phrase is generally a combination of the two: "probability of getting a positive result by chance when there is no effect"; but that's so long that people shorten it.)
[ Parent | on HN ]
Re:Registered clinical trials make positive findings vanish by Alex3917 on Tuesday August 18, 2015 @07:31PM (#10082870)
There are lots of reasons you could get a false positive result that have nothing to do with chance: e.g. non-representative sample, miscalibrated equipment, biased methodology, researcher fraud, flawed statistical analysis, etc.

C.f. http://blog.minitab.com/blog/adventures-in-statistics/how-to...

[ Parent | on HN ]
Re:Registered clinical trials make positive findings vanish by rcthompson on Tuesday August 18, 2015 @07:41PM (#10082908)
Indeed, but the comment that you quoted was referring to the case where the methodology and p-values are sound, and the only issue is testing multiple hypotheses without correction.
[ Parent | on HN ]