News for hackers. Stuff that matters.

A comment on A/B testing gets misused to juice metrics in the short term. Parent comment

Re:A/B testing gets misused to juice metrics in the short term by travisjungroth on Tuesday July 12, 2022 @03:35PM (#32074645)
It's really easy for this to be noise from false negatives. On an A/A test with five guardrail metrics and a threshold of p>0.05, you'll get a false negative 22.6% of the time.
[ Parent | on HN ]

There are no comments on this comment yet.