News for hackers. Stuff that matters.
Re:What increases signups by 28%? ‘Watch a video’ or ‘Get in... by revorad on Thursday January 06, 2011 @07:23AM (#2075002)
Given that you were able to increase CTR, I don't know how you can be so sure that visitors were not missing noticing the button. And even if you had good reason to believe that, why didn't you just change the button text instead of replacing it with a new button?

Sorry, I'm probably coming across as nitpicking, but I'm just trying to understand how people use A/B testing.

Re:What increases signups by 28%? ‘Watch a video’ or ‘Get in... by paraschopra on Thursday January 06, 2011 @07:27AM (#2075011)
No, you are totally right. The way you describe is how an actual A/B test should be done. But in practice companies resort to making changes in A/B test which involve changing more than one variable (in this case button style and text). That's because they can't afford the time and traffic it will take to methodically test one factor at a time. Also, primary motivation for many companies is to increase conversions. Knowing and deducing what increased conversions is a secondary priority.

By the way, I did not do this A/B test. I just interviewed one of my customers who did this test.

Re:What increases signups by 28%? ‘Watch a video’ or ‘Get in... by revorad on Thursday January 06, 2011 @07:39AM (#2075032)
Ah, now I get it. Sorry, for some reason, I assumed you did the test for the customer.

I shouldn't doubt your A/B testing ninja skills.

The button test, even combining style and text, is still quite a small change to test. Don't your customers want to test more drastic changes?

Re:What increases signups by 28%? ‘Watch a video’ or ‘Get in... by paraschopra on Thursday January 06, 2011 @07:49AM (#2075052)
Yes, they do. See our case study library (we have more than 15 A/B tests): http://visualwebsiteoptimizer.com/case-studies.php