Re:Winning A/B results were not translating into improved us...
by pocp2 on Monday September 29, 2014 @11:19AM
(#8383263)
For a second I thought you were Evan Miller who wrote about the exact same thing: http://www.evanmiller.org/how-not-to-run-an-ab-test.html
[ Parent
| on HN ]
Re:Winning A/B results were not translating into improved us...
by pmiller2 on Monday September 29, 2014 @02:43PM
(#8384424)
No, I'm not him, but stuff like fixing a sample size in advance and not stopping tests early without careful analysis are things I learned in my stat classes. This stuff should be stressed in any good intro stats class covering hypothesis testing. (I was a math major in college, so I had all of 2 courses in mathematical statistics and 0 in experimental design. I didn't go to the best school, but my stats teacher was a former industry statistician focusing on quality control.)