News for hackers. Stuff that matters.
Re:On the pitfalls of A/B testing by noloqy on Sunday July 14, 2013 @03:13PM (#6041836)
His choice of words is not very fortunate. If webpage A has an underlying conversion ratio of X, and webpage B has an underlying conversion ratio of Y, then if X is not equal to Y, in theory we could always find evidence that X is "significantly" different from Y.

If X is close to Y, we need a very large sample to achieve statistical significance on the hypothesis that they're not equal, whereas if X and Y are far apart, it is likely that a small sample will already indicate that fact.

There are no comments on this comment yet.