News for hackers. Stuff that matters.
Re:Most Winning A/B Test Results are Illusory [pdf] by paraschopra on Monday February 24, 2014 @09:56AM (#7291068)
No matter which method you adopt, you cannot eliminate false positives entirely. You merely decrease / control the proportion of them.
Re:Most Winning A/B Test Results are Illusory [pdf] by martingoodson on Monday February 24, 2014 @10:55AM (#7291438)
To remove all doubt, your interpretation of the statistics is incorrect. In particular this sentence is demonstrably false: "They are directionally correct, [...] the business will still do better implementing the variation (v/s not doing anything)."