Re:How Optimizely Almost Got Me Fired (2014)
by TheLogothete on Sunday January 10, 2016 @03:18AM
(#10874657)
The most common place people get stuck in a/b testing is proper design of experiments, i.e. absolutely no methodology (i.e. proper blocking) at all. It's ridiculous how a/b testing is perpetuated to be of paramount importance. A/B testing would be one of the last things I would do, if I worked at a small ecommerce company, for example. What lead to the current state is venture-backed companies and "growth hackers" and "conversion optimization experts" selling false promises to people who haven't read a marketing textbook in their life.
A common place where people get stuck when A/B testing is in a local minimum or maximum. For example, instead of tweaking button colors someone should instead redesign the whole form. Talking to users can help with this, but users are notoriously bad at not knowing what they really want.