Re:Appsumo reveals its A/B testing secret: only 1 out of 8 t...
by paraschopra on Tuesday February 15, 2011 @06:42AM
(#2220881)
I'd be wary of concluding too much from other people's A/B tests. Yes, they serve as a good starting point but doing A/B tests yourself is still imperative. Case in point: Noah found out headlines didn't matter to his audience. But for many of our customers, a small change in headline does produce significant results.
Before people did A/B test, it was generally assumed on Facebook ads that the title was the second most important factor on CTR so people spent a lot of time tuning title copy.
After lots of people did A/B testing it turns out that title copy has almost no impact, people switched their title to Chinese (for a non-chinese speaking audience) without seeing any change. That "no significant difference" means that thousands of man hours can be spent elsewhere rather than copywriting ad titles.