Re:Eric Ries: Vanity Metrics vs. Actionable Metrics
by davidbnewquist on Tuesday May 19, 2009 @06:36PM
(#617758)
Despite seeming a little jargon heavy, I was able to distill out few useful ideas.
For example, say you're considering a cool new CAPTCHA for your registration page. The article would advise doing an "A/B split test" before phasing out the old CAPTCHA.
Such a test would involve creating an alternate registration page with the new CAPTCHA, and randomly directing x% of users to the alternate page. You could then obtain registration completion % from both groups, which would drive your decision to switch to fully switch to the new CAPTCHA.
For example, say you're considering a cool new CAPTCHA for your registration page. The article would advise doing an "A/B split test" before phasing out the old CAPTCHA.
Such a test would involve creating an alternate registration page with the new CAPTCHA, and randomly directing x% of users to the alternate page. You could then obtain registration completion % from both groups, which would drive your decision to switch to fully switch to the new CAPTCHA.