News for hackers. Stuff that matters.
Re:Most Winning A/B Test Results are Illusory [pdf] by ernopp on Monday February 24, 2014 @09:37AM (#7290963)
Full disclosure: I work for Qubit who published this white paper.

Maybe the confusion here is in tests which have a "true" effect and an "observed" effect. If an experiment has a true effect, then you have some chance to observe it, which is the power.

But false positives have by definition already been observed as winners (that's what false positives are), so there's no need to apply the factor of 0.8 to them.

There are no comments on this comment yet.