Re:How Not To Run An A/B Test
by Samuel_Michon on Thursday December 29, 2011 @05:34PM
(#3405715)
"One way around the proposed problem is to become much more educated about statistics"
Sounds like a plan. I feel dumb after reading that article.
Any ideas on where to start? I found some videos at Khan Academy [0], will those help me grasp the concepts discussed in this thread?
[0] http://www.khanacademy.org/video/statistics--the-average?pla...
There's nothing magic about 95%, it was a convenient heuristic for science and that's all. With the vast amounts of data points that a high traffic website will generate, reaching p < 0.001 should be not too difficult and a significance threshold of 99.9% will erase a lot of other statistical sins.