News for hackers. Stuff that matters.

A comment on Google Website Optimizer Case Study: Daily Burn, 20%+ Improvement. Parent comment

Re:Google Website Optimizer Case Study: Daily Burn, 20%+ Imp... by patio11 on Wednesday August 12, 2009 @11:49AM (#758345)
It is easy to believe the things you want to believe. A/B testing is useful for convincing you of the things you don't want to believe.

Example: I'm releasing my Rails A/B testing framework this Sunday, with a case study writeup taken from my site. My site currently allows people to defer signup for the trial by means of a guest login. That sort of option is popular here. I'm told it increases usability and user engagement, right?

Without ruining the surprise: that is a testable hypothesis. ;)

[ Parent | on HN ]
Re:Google Website Optimizer Case Study: Daily Burn, 20%+ Imp... by onreact-com on Wednesday August 12, 2009 @12:11PM (#758415)
A/B testing is great, no doubt. The lesson of solely simplifying is just not a great outcome of such a test. It's like driving a car for the first time and then telling everybody that it's faster than walking.
[ Parent | on HN ]