News for hackers. Stuff that matters.
Re:What works in e-commerce – A meta-analysis of online expe... by gwern on Wednesday June 21, 2017 @12:12PM (#14605042)
> isn't really the average results, but what is possible and the distribution among beneficial results only

No, the bad results also matter: you are still spending visitors and revenues in testing out bad variants, which is part of determining the costs and benefits. Even with a bandit approach, you incur logarithmic regret in the number of variants. And testing a bad variant is common: the best category, 'scarcity', has a 16% probability of the variant being harmful. A Value of Information calculation has to take into account the harm done while testing.

Re:What works in e-commerce – A meta-analysis of online expe... by Silhouette on Wednesday June 21, 2017 @02:48PM (#14606791)
(Hence the final sentence of my previous comment.)