News for hackers. Stuff that matters.
Re:"Mad Libs" Style Form Increases Conversion 25-40% by patio11 on Thursday February 25, 2010 @07:13PM (#1151975)
They could be more certain with a multivariate test. I count 2^7 = 64 possible variations (counting the madlib/form as just one point of difference), which would take an awful lot of traffic to discriminate between. So I sort of kind of understand the desire to wrap it up in one simple A/B test.

But you're right, of course, there is no reason to assume that the change was from the obvious of the 7 changes instead of from all (or one!) of the other six changes. Did anybody notice that the button call to action changed? I've seen A/B tests where that did 20% by itself. Did anybody notice that the header call to action changed? I've seen A/B tests where that did 20% by itself. Did anybody notice that they added the words "Thank you."? I've seen A/B tests where... actually, I haven't for "Thank you." (if you've got one, please share) But you get the general idea.

Re:"Mad Libs" Style Form Increases Conversion 25-40% by eru on Thursday February 25, 2010 @07:44PM (#1152045)
> I count 2^7 = 64 possible variations (counting the madlib/form as just one point of difference), which would take an awful lot of traffic to discriminate between.

A fractional factorial design (http://en.wikipedia.org/wiki/Fractional_factorial_design) of the experiments and other statistic techniques (http://en.wikipedia.org/wiki/Bayesian_experimental_design) may decrease the amount of traffic needed.