News for hackers. Stuff that matters.

A comment on p-Hacking and False Discovery in A/B Testing. Parent comment

Re:p-Hacking and False Discovery in A/B Testing by p10_user on Sunday July 22, 2018 @02:36AM (#17585546)
It’s done in academia too. Along with “remove the outlier data that messes up our p values”
[ Parent | on HN ]
Re:p-Hacking and False Discovery in A/B Testing by glup on Sunday July 22, 2018 @04:35PM (#17588790)
The difference is that the people in academia know that it is statistically unsound and choose to act unethically. I think the problem in A/B testing is that a lot of developers don't know it is unsound.
[ Parent | on HN ]
Re:p-Hacking and False Discovery in A/B Testing by carlmr on Monday July 23, 2018 @06:42AM (#17591794)
This is one of the cases where self-taught developers are usually not as good as those with a "proper" education. In uni you'll learn a lot of tangentially related stuff like p-hacking and design of experiment which a lot of people won't pick up when self-taught.
[ Parent | on HN ]
Re:p-Hacking and False Discovery in A/B Testing by whyever on Monday July 23, 2018 @07:17AM (#17591904)
It might make sense to remove systematic outliers if you know they are from non-statistical effects.
[ Parent | on HN ]