Re:The Low Base Rate Problem (2014)
by marginalia_nu on Thursday April 14, 2022 @12:41PM
(#31029422)
I took the GP to mean they wanted tooling to do design and interpret A/B tests for them rather than having to consider cases like false negatives and what have you, to reduce them down to something that's easier to understand.
My point is that the process of designing experiments and interpreting their outcome is nothing other than science.
Science is hard and goes much deeper than the trivial stuff like significance and power. Even scientists, who know all these formulae inside and out struggle with constructing experiments and interpreting the outcome.
To properly think about experimental outcomes, you need to actually have a firm grasp of statistics. The moment you try to convert it into a message that doesn't require understanding statistics is the moment it stops being informative.
My point is that the process of designing experiments and interpreting their outcome is nothing other than science.
Science is hard and goes much deeper than the trivial stuff like significance and power. Even scientists, who know all these formulae inside and out struggle with constructing experiments and interpreting the outcome.
To properly think about experimental outcomes, you need to actually have a firm grasp of statistics. The moment you try to convert it into a message that doesn't require understanding statistics is the moment it stops being informative.