Re:Most AB-tests will fail
by m4xt3r on Wednesday June 19, 2013 @11:12AM
(#5905979)
#2 is totaly relevant to our Macro Recorder app.
Macro Recorder is really popular and most of the visitors are already "pre-sold". They don't care about the landing page.
We used to run tests on that page, because it has 1k daily visitors, we thought that we could test ideas more quickly. That was before we realised, that we can't rely on the results.
We even used false conclusions, based on those results, to make changes to landing pages for other products. Perhaps the biggest mistake we've made.
Macro Recorder is really popular and most of the visitors are already "pre-sold". They don't care about the landing page.
We used to run tests on that page, because it has 1k daily visitors, we thought that we could test ideas more quickly. That was before we realised, that we can't rely on the results.
We even used false conclusions, based on those results, to make changes to landing pages for other products. Perhaps the biggest mistake we've made.