Re:Optimizely (YC W10) Increases Homepage Conversion Rate by...
by StavrosK on Thursday June 09, 2011 @05:31AM
(#2636518)
Were you looking at the statistics all the time to see when they passed 90% confidence? That invalidates them, just so you know.
Basically, if you're watching it, it won't science.
Re:Optimizely (YC W10) Increases Homepage Conversion Rate by...
by petercooper on Thursday June 09, 2011 @09:53PM
(#2639487)
I'm not sure how tongue in cheek that was ;-) but 90% confidence was not a goal, at least. I just got bored after a week and wanted to move on. C'est la vie..
I ran a 5 way split test for 9 days on my newsletter's signup page (JavaScript Weekly). My original page was the 2nd best performing but an identical page just without the subscriber count got a 8% higher conversion (or about 20% more signups in all) with a 90% confidence at the end of testing.
The worst performer? A signup page with no screenshot preview of the newsletter. Sent conversions from about 37% down to a mere 3% (!!) Lesson learned? Always have visuals or screenshots on pages where you're trying to get people to sign up for things they aren't sure about.