Re:A/B testing original design vs. random template bought fr...
by chiefalchemist on Friday March 15, 2019 @12:49PM
(#19401496)
Small sample size aside, they used the wrong KPI (imho). This needs to be tested over time. That is, which one retains better? Focusing on conversion's is a fool's errand. That is, you convert better but then if churn is higher you could end up at net loss. That's not ideal, obviously.
I think also, you'd have to look at referrals (if you have them). Perhaps a lower subscription rate actually led to more sales because (for some reason) those subscribers like to tell their friends.
I think also, you'd have to look at referrals (if you have them). Perhaps a lower subscription rate actually led to more sales because (for some reason) those subscribers like to tell their friends.
The analysis here is too shallow.