Launch HN: Just words (YC W24) – Optimize your product's copy for user growth

Re:Launch HN: Just words (YC W24) – Optimize your product's ... by nehagetschoice on Monday March 04, 2024 @06:51PM (#39597688)

We either run experiments with higher traffic diverted to treatment (eg: 50% instead of 1%), or run it for a longer time. This is the reason why we need companies to have at least 2K monnthly users to run pilots with us.

[ Parent | on HN ]

Re:Launch HN: Just words (YC W24) – Optimize your product's ... by earthling998 on Monday March 04, 2024 @07:00PM (#39597746)

Thanks! Cool idea on that front and in general; fun to see people taking copy seriously!

I am still a little fuzzy on how this might play out on a real site, since lots of monthly users doesn’t always mean lots of conversions (especially for one-time, high dollar transactions).

Let’s say I have a page that gets 100k unique visitors per month. I show them 5 different variants of a “nudge” widget. Some do better than others, but they all hover at <1% CTR.

While there may be a story to tell as far as winners and losers (e.g. mobile users converted to variant A at 2x the rate of desktop users), how do you confidently report a “winner” from what amounts to a few thousand conversions in a month?

[ Parent | on HN ]