Re:A/B testing gets misused to juice metrics in the short term
by throwaway290 on Tuesday July 12, 2022 @10:13AM
(#32070256)
They are always different. You cannot hold a conclusive A/B test for customer experience.
Search engine, a single-purpose tool, is as simple as they come regarding customer experience. Still, a good search algorithm can make me click on the first result if it is good, and a bad search algorithm can make me click the first result because they are so bad that scrolling further is a waste of time, especially if I already needed to scroll through widgets and ads to get to the first result.
It's not about just selecting good metrics, it's about higher level picture that A/B testing can never get you.
Search engine, a single-purpose tool, is as simple as they come regarding customer experience. Still, a good search algorithm can make me click on the first result if it is good, and a bad search algorithm can make me click the first result because they are so bad that scrolling further is a waste of time, especially if I already needed to scroll through widgets and ads to get to the first result.
It's not about just selecting good metrics, it's about higher level picture that A/B testing can never get you.