A/B testing original design vs. random template bought from ThemeForest

Re:A/B testing original design vs. random template bought fr... by codetrotter on Friday March 15, 2019 @11:45AM (#19400720)

They said the following in the beginning though:

> For example if you want to test a tweak that results in 5% more conversions, you need about 3000 sales to detect it! For Candy Japan this would mean waiting for about 10 years for the test to complete.

But they want to do something still to try and improve sales. Seems reasonable even if not scientific.

[ Parent | on HN ]

Re:A/B testing original design vs. random template bought fr... by cle on Friday March 15, 2019 @11:57AM (#19400836)

This is not the right approach to take then. There are lots of other approaches to decision making outside of hypothesis testing, use them! This is not an appropriate use of hypothesis testing and can very much lead you toward making the WRONG decision.

For example, with such small numbers, there isn't much value in aggregate statistics. It would take a day or two to go through each one individually and see what happened, and you'd probably learn way more about your customers.

[ Parent | on HN ]