A/B testing gets misused to juice metrics in the short term

Re:A/B testing gets misused to juice metrics in the short term by dpe82 on Tuesday July 12, 2022 @10:54AM (#32070885)

In fact, you want the dismiss button to be easily discovered and used. Dismissals are an important signal about the quality of the content; just as important as clicks. When you make the button impossible to use you rob yourself of that signal while simultaneously making click data far less reliable.

[ Parent | on HN ]

Re:A/B testing gets misused to juice metrics in the short term by bbarnett on Tuesday July 12, 2022 @11:16AM (#32071187)

Unless you're a manager, not getting the results desired.

Then the problem is people, and "they're just clicking close out of reflex!".

Cue hiding this, and results you wanted appear! Success! Raise! Promotion! Or, maybe more funding, due to signs of greater engagement.

[ Parent | on HN ]

Re:A/B testing gets misused to juice metrics in the short term by dpe82 on Tuesday July 12, 2022 @05:05PM (#32075676)

Sure; until a company that actually understands this stuff comes along and eats that company's lunch. In the long run, reality eventually wins.

[ Parent | on HN ]