News for hackers. Stuff that matters.
Re:Landing pages that convert by patio11 on Saturday June 12, 2010 @02:52AM (#1425672)
slaps forehead User counts. I have them on my purchasing page, why not put them on my landing page.

I love little ideas like that where I can have the A/B test literally live within a minute.

With regards to testimonials: I have rotated through a couple. They're all authentic so I can't create perfect tests of every element, but my experience has been that a generic testimonial attributed to someone just like the target customer outperforms a better testimonial attributed to someone who isn't. (In my case, a name suggesting an older woman clocks a generic male name. Which in unsurprising with 95% of my customers being ladies and most of them being older.)

Re:Landing pages that convert by jules on Saturday June 12, 2010 @06:37AM (#1425809)
How much difference did no testimonial vs male testimonial vs female testimonial make?
Re:Landing pages that convert by patio11 on Saturday June 12, 2010 @01:10PM (#1426242)
A statistically significant difference and then a statistically significant difference. A/B tests don't guarantee that the observed magnitude of the difference is meaningful.

(To satisfy your curiosity, it was Glenda 25.12% vs Dave 23.38%, significant at 90% confidence, and I don't have numbers handy for the testimonial vs. no testimonial test. That was prior to me tracking the results in a systematic fashion. It doesn't look like it is written anywhere on my blog, either.)

Re:Landing pages that convert by jules on Saturday June 12, 2010 @03:23PM (#1426396)
There seems to be a tension between running the test longer and collecting more profits. If you run the test longer you are less likely to pick the bad option, at the expense of trying the bad option for a longer period of time. Have you thought about a mathematical solution to this problem? Perhaps there is a good way to decide when to stop automatically.
Re:Landing pages that convert by davidblair on Saturday June 12, 2010 @08:35PM (#1426755)
What do you use as your p-value cutoff?

I've heard many arguments for .05 being the highest acceptable for research but it seems like you have found an acceptable p-value that is out of that range by quite a bit.

Re:Landing pages that convert by bryanh on Saturday June 12, 2010 @10:02PM (#1426834)
I figure that if the test is at 90% confidence, it usually means we can't really make a really really bad decision going with the favorite. So what if that tiny increase in conversions wasn't really there, I figure its time to move on to another, possibly more lucrative test. Maybe Patrick has a different or more fleshed out opinion on this.