News for hackers. Stuff that matters.
Re:Most Winning A/B Test Results are Illusory [pdf] by tristanz on Sunday February 23, 2014 @05:56PM (#7288069)
Putting aside bandits and all that, it seems like the first step should be to set up a hierarchical prior which performs shrinkage. Multiple comparisons and stopping issues are largely due to using frequentist tests rather than a simple probabilistic model and inference that conditions and the observed data.

Gelman et al, "Why we (usually) don't have to worry about multiple comparisons" http://arxiv.org/abs/0907.2478

There are no comments on this comment yet.