News for hackers. Stuff that matters.
Re:Conservation of Intent: why A/B tests aren’t as effective... by srean on Wednesday July 04, 2018 @07:32AM (#17456599)
I think you might be missing btilly's point. He is saying that the testing protocol should factor in the cost of being wrong. These costs might be drastically different for a business and for a scientific pursuit.

If A is ahead of B by a hair and the my flawed protocol chooses B the cost to business might not be that high. But the same protocol might not be a good one if the cost of making that mistake is very high. The probabilities of the errors remain the same for the two scenarios, the expected costs are different.

There are no comments on this comment yet.