earthling998

Thanks! Cool idea on that front and in general; fun to see people taking copy seriously!

I am still a little fuzzy on how this might play out on a real site, since lots of monthly users doesn’t always mean lots of conversions (especially for one-time, high dollar transactions).

Let’s say I have a page that gets 100k unique visitors per month. I show them 5 different variants of a “nudge” widget. Some do better than others, but they all hover at <1% CTR.

While there may be a story to tell as far as winners and losers (e.g. mobile users converted to variant A at 2x the rate of desktop users), how do you confidently report a “winner” from what amounts to a few thousand conversions in a month?

How do you confidently report a “best version” on lower traffic sites where each variant may only see a couple dozen successes per day?