Re:Estimating the chances of something that hasn’t happened yet
by hinkley on Thursday November 15, 2018 @07:47PM
(#18465124)
Statistics is a tricky beast. If 3 runs gives you a 95% chance that your change is an improvement, and you use this process on ten different improvements, there's a 40% chance that at least one of them is bogus.
[ Parent
| on HN ]
Re:Estimating the chances of something that hasn’t happened yet
by hinkley on Friday November 16, 2018 @02:59PM
(#18471517)
Would love to see a refutation from a downvoter.
Re:Estimating the chances of something that hasn’t happened yet
by agonmon on Sunday November 18, 2018 @04:45PM
(#18482682)
I believe you have it right.
Odds that all changes are improvements in 10 changes is: (.95)^10
Odds that at least one change is not improvement is:
1-(.95)^10 = 0.40
One could probably make an argument that not all trials are independent, but not sure what else down-voters saw.