Why Multi-armed Bandit algorithms are superior to A/B testing (with Math)

Re:Why Multi-armed Bandit algorithms are superior to A/B tes... by btilly on Sunday June 03, 2012 @09:42PM (#4062319)

Even if you're making larger and more complex changes, the overhead of your testing methodology remains the same. That is how you measure things should be a fixed (small) effort, The cost of building the test is whatever the test is.

In other words multi-armed bandit versus A/B test is something that you shouldn't be deciding based on the effort of the testing methodology.

[ Parent | on HN ]

Re:Why Multi-armed Bandit algorithms are superior to A/B tes... by polyfractal on Monday June 04, 2012 @08:12AM (#4063620)

I don't think he was referring to the technology behind the A/B test itself, but rather the technology behind the change that was being made.

That's how I interpreted his statement. I agree with you that the actual A/B testing overhead should be minimal and fairly trivial to put into place.

[ Parent | on HN ]