News for hackers. Stuff that matters.
Re:Most Winning A/B Test Results are Illusory [pdf] by hvass on Sunday February 23, 2014 @06:03PM (#7288104)
I strongly recommend using Evan Miller's free A/B testing tools to avoid those issues!

Use them to really know if conversion rate is significantly different, whether the mean value of two groups is significantly different and how to calculate sample size:

http://www.evanmiller.org/ab-testing/

Re:Most Winning A/B Test Results are Illusory [pdf] by napoleond on Sunday February 23, 2014 @08:46PM (#7288664)
This is awesome, thanks for the link! (And the visualizations help a ton, especially for the t-test... it's been a while since I took any stats courses and the terminology always puts me off a bit but the graphs make sense.)
Re:Most Winning A/B Test Results are Illusory [pdf] by lingben on Monday February 24, 2014 @09:49AM (#7291034)
thanks but what does "expected conversion rate" mean exactly? it isn't defined and I couldn't find that term anywhere else on the site. EDIT: ah, ok, got it. but why is their default expected conversion rate set so high? sheeesh

most people have conversion rates between 1-3%