News for hackers. Stuff that matters.
Re:A Bayesian Approach to A/B Testing by ced on Saturday May 05, 2012 @07:33PM (#3933622)
I've looked at your website, and from what I gathered, I would make the same criticism as I made for Anscombe's rule: it's not easy at all to decide what rewards should be, and how to put a price on exploration vs. exploitation. The more I think about it, the more I feel that an engineer looking at Beta distributions could weigh the trade-offs and make a better decision than a black-box algorithm with inadequate assumptions.

Granted, this doesn't really scale to testing many combinations of feature, and I think that I can see what you're shooting for. Best of luck with Myna.

There are no comments on this comment yet.