News for hackers. Stuff that matters.
Re:How Flightfox doubled their remarketing email conversions by acabal on Tuesday October 09, 2012 @09:11AM (#4631285)
It doubled, or in this case, increased by 1.6%. That's nice, but how can you know that a 1.6% increase is a direct result of these tweaks, instead of just natural ups and downs in email response? Maybe 1.6% of recipients were having an unnaturally good day and decided to click that month? That's only 1 in 98 people after all. If it had been 10% or something like that then that's an obvious difference, but 1.6%?
Re:How Flightfox doubled their remarketing email conversions by wolffnc3 on Tuesday October 09, 2012 @11:32AM (#4632079)
Typically A/B tests are run until they reach a statistically significant result. The article doesn't include the sample size, but with a sample size as small as 500 users for each test an increase from 1.6% to 3.2% would be a change larger than 1 standard deviation, and thus can be assumed to be more than just random noise.