News for hackers. Stuff that matters.
Re:How Not To Run An A/B Test by shalmanese on Thursday December 29, 2011 @02:40PM (#3405009)
One way around the proposed problem is to become much more educated about statistics, another way is just to bump your threshold of statistical significance up to 99.9%.

There's nothing magic about 95%, it was a convenient heuristic for science and that's all. With the vast amounts of data points that a high traffic website will generate, reaching p < 0.001 should be not too difficult and a significance threshold of 99.9% will erase a lot of other statistical sins.

Re:How Not To Run An A/B Test by Samuel_Michon on Thursday December 29, 2011 @05:34PM (#3405715)
"One way around the proposed problem is to become much more educated about statistics"

Sounds like a plan. I feel dumb after reading that article.

Any ideas on where to start? I found some videos at Khan Academy [0], will those help me grasp the concepts discussed in this thread?

[0] http://www.khanacademy.org/video/statistics--the-average?pla...