20 lines of code that beat A/B testing every time

Re:20 lines of code that beat A/B testing every time by mattj on Tuesday May 29, 2012 @07:15PM (#4040489)

This is actually pretty straightforward to overcome (and one of the real strengths of the bayesian approach). Rather than using the direct counts of success for each group, add a prior belief that americans will favor baseball and non-americans will favor soccer (you can experiment to determine this number).

You can now evaluate the results conditioned on each group (american / non-american).

[ Parent | on HN ]

Re:20 lines of code that beat A/B testing every time by nickzoic on Tuesday May 29, 2012 @07:48PM (#4040596)

Or if you don't want to add in a prior specific belief about americans vs. non-americans, just keep one counter per option per continent-of-source-IP (or whatever) and the learning algorithm should work it out on its own. Of course, if you use too many bins then learning is going to take far too long.

[ Parent | on HN ]