News for hackers. Stuff that matters.
Re:Most Winning A/B Test Results are Illusory [pdf] by darkmighty on Sunday February 23, 2014 @08:15PM (#7288566)
There are also some complex problems with assumptions that are infrequently addressed, e.g. maybe if a regular user sees a structural/cosmetic change he is more likely to look at that and click it, while that would fade away in steady-state.
Re:Most Winning A/B Test Results are Illusory [pdf] by RyJones on Sunday February 23, 2014 @10:17PM (#7288915)
True. We did everything we could to account for that, though. Someone chooses to clear cookies every time they load a page - not much we could do.