News for hackers. Stuff that matters.

A comment on A/B testing original design vs. random template bought from ThemeForest. Parent comment

Re:A/B testing original design vs. random template bought fr... by olavolav on Friday March 15, 2019 @08:05AM (#19398867)
Whatever test you do, you would need to know the total number of visitors in each group, right?

And unless I missed it the article doesn't state those numbers.

Intuitively, the numbers you quoted would be more significant the bigger the test and control groups are.

[ Parent | on HN ]
Re:A/B testing original design vs. random template bought fr... by kqr on Friday March 15, 2019 @12:48PM (#19401489)
That's not very intuitive for me. Let's do some limit analysis: imagine the groups were one million sessions each, but the convertions in the groups were only one and two people respectively. Wouldn't this result seem like the result of random chance?

The conversion rate is basically one in a million in both cases.

[ Parent | on HN ]