News for hackers. Stuff that matters.
Re:The Low Base Rate Problem (2014) by xpe on Thursday April 14, 2022 @12:15PM (#31028975)
Yes, in some cases, teams want an experiment to back them up.

If a decision maker cannot dedicate the necessary resources to properly power a test, it arguably would be more rational to use another process to make the decision. Perhaps a thoughtfully designed team discussion or even a 'Person P will make the decision because the uncertainty is relatively high, consensus is relatively low, and additional discussion is unlikely to clarify -- meanwhile we could go ahead and try the thing and see what happens!"

Re:The Low Base Rate Problem (2014) by tqi on Thursday April 14, 2022 @01:11PM (#31029925)
> If a decision maker cannot dedicate the necessary resources to properly power a test

Apologies if this is what you meant, but in a lot of cases it's not a matter of not dedicating enough resources so much as there aren't enough resources (ie visitors, users, posts, etc) to detect reasonable effect size. And in those cases, it seems like the author's point is that running a test is actively worse than a gut call or focus group or whatever else. But to your point, product teams often don't like gut calls because they feel subjective and unilateral, so having a test allows some degree of shared accountability for decisions.

Re:The Low Base Rate Problem (2014) by xpe on Thursday April 14, 2022 @03:55PM (#31032449)
Well said.