News for hackers. Stuff that matters.
Re:Conservation of Intent: why A/B tests aren’t as effective... by kolpa on Tuesday July 03, 2018 @06:15PM (#17454005)
Truth isn't that different from business. Both are statistical. Making a decision that is not statistically valid is bad business, as you could be making things worth as often as better.
Re:Conservation of Intent: why A/B tests aren’t as effective... by srean on Wednesday July 04, 2018 @07:32AM (#17456599)
I think you might be missing btilly's point. He is saying that the testing protocol should factor in the cost of being wrong. These costs might be drastically different for a business and for a scientific pursuit.

If A is ahead of B by a hair and the my flawed protocol chooses B the cost to business might not be that high. But the same protocol might not be a good one if the cost of making that mistake is very high. The probabilities of the errors remain the same for the two scenarios, the expected costs are different.