News for hackers. Stuff that matters.
Re:A/B testing original design vs. random template bought fr... by chiefalchemist on Friday March 15, 2019 @12:49PM (#19401496)
Small sample size aside, they used the wrong KPI (imho). This needs to be tested over time. That is, which one retains better? Focusing on conversion's is a fool's errand. That is, you convert better but then if churn is higher you could end up at net loss. That's not ideal, obviously.

I think also, you'd have to look at referrals (if you have them). Perhaps a lower subscription rate actually led to more sales because (for some reason) those subscribers like to tell their friends.

The analysis here is too shallow.

There are no comments on this comment yet.