News for hackers. Stuff that matters.
Re:A/B Testing Rigorously (without losing your job) by mistercow on Thursday January 24, 2013 @11:47PM (#5114013)
That is very true. However, you do have much better reason to believe that the option is neutral than you have to believe that the option is beneficial. In an example like the one given in the article, you also likely have enough statistical power to be reasonably confident that the option is close to neutral, so if you're making a negative decision, it's probably not strongly negative.
Re:A/B Testing Rigorously (without losing your job) by prosa on Friday January 25, 2013 @02:10AM (#5114420)
Right. And you can also ratchet up the statistical power you want, at the cost of increased sample size requirements.