News for hackers. Stuff that matters.
Re:Do data-driven companies win? by jlei523 on Friday July 29, 2022 @05:31AM (#32274803)
A/B testing can definitely make product teams feel like they're making progress but it should not be a team's only strategy.

It's a bit like moving the furniture around the same room.

Re:Do data-driven companies win? by cestith on Friday July 29, 2022 @09:52AM (#32276874)
Nobody has a big enough test set to A/B test every combination of elements for a web page, a product feature set, or even a conference presentation. The combinatorics are just unwieldy.

Someone has to have multiple good ideas and the ability to carry them out properly before an A/B test is even valuable. Otherwise you're measuring one uninformed random change against another. A/B confirms whether you've succeeded in improving something. It can't really suggest what to try next.