News for hackers. Stuff that matters.
Re:Expedia on how one extra data field can cost $12m by samd on Tuesday November 02, 2010 @08:20AM (#1860101)
I don't know what's more surprising, that one small change is worth millions or that A/B testing is still news to some companies.
Re:Expedia on how one extra data field can cost $12m by patio11 on Tuesday November 02, 2010 @09:12AM (#1860262)
I have spoken to a lot of companies who have A/B testing somewhere on their radar, but some combination of "That's nobody's job, so we haven't actually started doing it", (sometimes) organizational infighting (marketing wants to start, engineering has higher priorities and refuses to allocate resources to "twiddling text", etc), and uncertainty about outcomes means they don't start it yet.

One of my big tasks at some consulting clients is convincing people to buy into A/B testing. The easy way is to show them how their website is leaking staggering sums of money, and present A/B testing as the cheap, reliable solution to that problem in the future.

These are smart software companies who transact most of their business online, by the way. The broader economy which isn't primarily web-focused has many, many, many opportunities to improve their execution.

Re:Expedia on how one extra data field can cost $12m by brianbreslin on Tuesday November 02, 2010 @08:38AM (#1860157)
Wasn't there a famous post a few years back by amazon about their a/b tests resulting in 60million in sales increases, just by changing the button colors?

Everyone working on ecommerce sites of this scale should be expected to know about this stuff

Re:Expedia on how one extra data field can cost $12m by njharman on Tuesday November 02, 2010 @11:18AM (#1860755)
This wasn't A/B testing. This was fixing UI mistakes based on UX analysis. There was no 'B'. They weren't comparing A and B.
Re:Expedia on how one extra data field can cost $12m by samd on Tuesday November 02, 2010 @05:08PM (#1862358)
UX analysis and A/B testing go hand in hand. There'd be no way to know that the Company field was a UI mistake if it had not been tested.
Re:Expedia on how one extra data field can cost $12m by CapitalistCartr on Tuesday November 02, 2010 @11:33AM (#1860827)
In this case, the A/B testing would have been with that field and without.
Re:Expedia on how one extra data field can cost $12m by pavel_lishin on Tuesday November 02, 2010 @01:06PM (#1861302)
How would you know which fields to use?
Re:Expedia on how one extra data field can cost $12m by leftnode on Tuesday November 02, 2010 @09:16AM (#1860272)
Yup, it's shocking. However, is this more of a problem with A/B testing or just bad user experience/design? I suppose an A/B test would've been to remove the Company field from the payment process and see how things have performed.

I think companies are more concerned with more obvious A/B tests (button color, page headings) than with testing large changes to their site or testing more in depth things like user experience.

Re:Expedia on how one extra data field can cost $12m by aplusbi on Tuesday November 02, 2010 @11:23AM (#1860780)
A/B testing probably wouldn't have caught this.
Re:Expedia on how one extra data field can cost $12m by matwood on Tuesday November 02, 2010 @09:45AM (#1860356)
I don't know what's more surprising, that one small change is worth millions or that A/B testing is still news to some companies.

Yeah, my company has been doing A/B (we call it Champion/Challenger) testing for years.

Re:Expedia on how one extra data field can cost $12m by thibaut_barrere on Tuesday November 02, 2010 @09:21AM (#1860288)
A/B anything (testing or dual projects at once etc) is seen in some places as a "lack of skills, otherwise you would know beforehand".

(which I don't agree with, in case someone doubts!)