News for hackers. Stuff that matters.
Re:Writing Decisions: Headline tests on the Highrise signup ... by aneesh on Tuesday January 13, 2009 @09:26PM (#433062)
One caveat in A/B tests -- you have to be careful in deciding what metric you use to pick the winner. They used the number of immediate conversions. But maybe they just increased signups of freeloaders who will abandon the account within 30 days. And maybe not. Another, possibly better, metric would be the number of paying customers after 30 days, rather the number of conversions today.

There are no comments on this comment yet.