Re:Why small tweaks and split testing don't work, and what t...
by hammock on Thursday November 29, 2018 @02:53PM
(#18563338)
Hi, I highly doubt booking.com's success came from CTA button colors - or any testing on the consumer side of the platform, for that matter. Their success is rooted in how they are able to take advantage of hotels and their distribution power.
[ Parent
| on HN ]
Re:Why small tweaks and split testing don't work, and what t...
by hn_throwaway_99 on Thursday November 29, 2018 @03:09PM
(#18563480)
This is absolutely false, and anyone who works at booking.com will tell you this. https://taplytics.com/blog/how-booking-comss-tests-like-nobo...
Re:Why small tweaks and split testing don't work, and what t...
by hammock on Thursday November 29, 2018 @06:02PM
(#18564990)
Because they test a lot doesn't mean it's critical to their success. That's a cargo cult fallacy.
Your article points to 2-3x industry average conversion rates (for existing traffic), but says nothing about the more important factors of acquisition or inventory.
Re:Why small tweaks and split testing don't work, and what t...
by hn_throwaway_99 on Thursday November 29, 2018 @08:39PM
(#18565873)
Except that Booking actually has data to back this all up. It's very common to do "holdbacks" so that you, say, give an old version of your website to a very small portion of your user's, then compare its conversion rate to a version of your site that has all the latest A/B test winners. Booking also tracks religiously how conversion rates change over time.
The whole point being that the way Booking (and other) companies act is the exact opposite of a cargo cult. Everything needs to be backed up with data and challenging conclusions is part of the culture.
All that said, a common complaint of Booking is that it has a ton of "dark UI" patterns, so it will be interesting if there is any long term blowback against short term A/B winners that erode the goodwill to the brand over time.