A/B testing gets misused to juice metrics in the short term

Re:A/B testing gets misused to juice metrics in the short term by kjhgkjghkj on Tuesday July 12, 2022 @09:18AM (#32069471)

Intentional or not, one outcome on sites that are relentlessly A/B tested is that the resulting UI design lets users know that content they want is there, they just need to click and scroll a bit more to find it.

Having left FB years ago, I now watch people "navigate" their site/apps with disbelief.

[ Parent | on HN ]

Re:A/B testing gets misused to juice metrics in the short term by regularfry on Tuesday July 12, 2022 @09:32AM (#32069648)

Isn't that exactly the problem? The resulting UI isn't designed, it's aggregated across a disjointed set of granular tweaks.

[ Parent | on HN ]

Re:A/B testing gets misused to juice metrics in the short term by ssharp on Tuesday July 12, 2022 @09:40AM (#32069761)

This is part of the "unchecked" part of AB testing the headline mentions.

You, of course, need to ensure the granular tweaks can be rolled up into something usable as the granular tweaks prove successful. You can't just keep bolting on UI changes while losing sight of the larger experience. Each incremental A/B test is testing against a previously successful variant so eventually the control is radically different from where it started and you're only concerned about beating the control. Using a longer-term holdout group or reseting the control experience during incremental testing can help mitigate this and get you zoomed out a bit from the local maxima.

[ Parent | on HN ]

Re:A/B testing gets misused to juice metrics in the short term by kjhgkjghkj on Tuesday July 12, 2022 @09:38AM (#32069728)

A problem for who? Given that people already invested in the product ecosystem seem to have almost limitless patience to scroll for the right content, I'm sure it improves almost every user time and attention metric.

It's why I saw it as my moral duty to leave (as well as the other FB properties), so that at least in a small way, I "produce content" that is only available by interacting with me as a person.

[ Parent | on HN ]