Re:Conservation of Intent: why A/B tests aren’t as effective...
by kolpa on Tuesday July 03, 2018 @06:15PM
(#17454005)
Truth isn't that different from business. Both are statistical. Making a decision that is not statistically valid is bad business, as you could be making things worth as often as better.
[ Parent
| on HN ]
Re:Conservation of Intent: why A/B tests aren’t as effective...
by srean on Wednesday July 04, 2018 @07:32AM
(#17456599)
I think you might be missing btilly's point. He is saying that the testing protocol should factor in the cost of being wrong. These costs might be drastically different for a business and for a scientific pursuit.
If A is ahead of B by a hair and the my flawed protocol chooses B the cost to business might not be that high. But the same protocol might not be a good one if the cost of making that mistake is very high. The probabilities of the errors remain the same for the two scenarios, the expected costs are different.