News for hackers. Stuff that matters.
Re:How Optimizely Almost Got Me Fired (2014) by yummyfajitas on Saturday January 09, 2016 @10:31PM (#10874015)
This is necessary with textbook statistical methods which are explicitly based on the assumption of looking at exactly one time. However, modern tools (Optimizely, VWO, Qubit) do not require this. [1]

If you use one of these tools it's completely safe to run tests until the test says win.

There are still caveats to account for the real world - e.g., only stop the test after an integer number of weeks - but statistically this is a solved problem.

[1] The blog post under discussion was written before Optimizely's StatsEngine and VWO's SmartStats.

There are no comments on this comment yet.