News for hackers. Stuff that matters.

A comment on A/B testing gets misused to juice metrics in the short term. Parent comment

Re:A/B testing gets misused to juice metrics in the short term by jl6 on Tuesday July 12, 2022 @01:03PM (#32072816)
True, A could be “annoy users” and B could be “don’t annoy users”, and A could perform better overall, but in this framework you might be missing C which is “annoy users except those already deep in the funnel”.
[ Parent | on HN ]
Re:A/B testing gets misused to juice metrics in the short term by tshaddox on Tuesday July 12, 2022 @01:31PM (#32073223)
My point is more that there can be two groups of users with mutually exclusive desires, and it can be practical to choose to satisfy one group over the other based on your particular goals. There's not always some monotonically increasing function where you can over time satisfy a higher and higher portion of your users.
[ Parent | on HN ]