Re:Solving Algorithmic Problems in Python with Pytest (2019)
by smallnamespace on Saturday December 19, 2020 @10:41AM
(#25478039)
In practice there is no such tradeoff, the tails of a Gaussian fall off extremely rapidly so if you simply 10x the number of iterations you’ll have failure become astronomically unlikely.
Re:Solving Algorithmic Problems in Python with Pytest (2019)
by daveFNbuck on Saturday December 19, 2020 @10:41AM
(#25478042)
You can safely use this type of test in your pipeline if you set a fixed random seed.
For example, if the test flips the coin 1,000 times, there is better than a 99% chance that the outcome would result in 450 to 550 heads. So if you write the test using `450 <= heads <= 550`, you know that it will fail ~1% of the time. And if you expand the range to reduce the rate of false negatives, you have reduced the confidence the test is validating correctness.
Having said that, I still find statistical tests to be very helpful when building out code that uses randomness. However, these tests typically do not make it into the CI/CD pipeline.