Re:Show HN: ABBA - Beautiful, open-source A/B test calculator
by matsiyatzy on Thursday March 22, 2012 @05:50AM
(#3739066)
I would perhaps also have added a calculation of the power of the test. But I really love that they're actually explaining the details behind it, which a lot of other A/B-test calculators out there don't do.
Re:Show HN: ABBA - Beautiful, open-source A/B test calculator
by gostevehoward on Wednesday March 21, 2012 @10:41PM
(#3738090)
Theirs has a nice, simple presentation which makes it very easy to use, especially for casual users. But it includes much less information (no confidence intervals and p-values are only approximate), uses much simpler statistics (z-test with no multiple testing correction can result in highly overconfident results), and doesn't describe what underlying statistics it uses. Although not as immediately accessible, I think Abba has much more to offer to anyone who's serious about A/B testing.
We also have a blog post that explains mathematics of A/B testing: http://visualwebsiteoptimizer.com/split-testing-blog/ab-test...
This is something like 25 lines of code.