News for hackers. Stuff that matters.
Re:Stop A/B Testing and Make Out like a Bandit by charliepark on Monday August 01, 2011 @07:43AM (#2831617)
I could be misreading it, but I believe the premise is "more traction => more action".

If I'm right, the idea is that the algorithm dynamically weights the "display frequency" of the two (or n) options. So as one of the a/b options shows itself to be more successful, it's shown more frequently. Because the test is self-correcting, you as an A/B test runner don't have to decide when the results are significant enough, and the program itself will automatically choose the more successful option.

Re:Stop A/B Testing and Make Out like a Bandit by datadon on Monday August 01, 2011 @08:14AM (#2831722)
Going purely off of your boiled down description, this seems like a more advanced version of Genetify (https://github.com/gregdingle/genetify/wiki).

@noelwelsh - care to comment on now what you're doing at Untyped differs from what Genetify offers?

Re:Stop A/B Testing and Make Out like a Bandit by e-dard on Monday August 01, 2011 @08:51AM (#2831853)
The difference in performance between using evolutionary approaches and bandit algorithms is complex and problem dependent. There is no "one algorithm is better than the other".

I've used both approaches for a different application and neither dominate the other.

Re:Stop A/B Testing and Make Out like a Bandit by StavrosK on Monday August 01, 2011 @08:32AM (#2831789)
That looks like it uses genetic algorithms, which is much less optimal and more exploratory than bandit algorithms. What noelwelsh is proposing would lead to better results much more quickly and without random permutations of elements.
Re:Stop A/B Testing and Make Out like a Bandit by noelwelsh on Monday August 01, 2011 @08:40AM (#2831817)
I agree :)
Re:Stop A/B Testing and Make Out like a Bandit by noelwelsh on Monday August 01, 2011 @07:44AM (#2831623)
That's correct.