Re:Continuous Deployment at IMVU: Doing the impossible fifty...
by ivey on Tuesday February 10, 2009 @10:44AM
(#475282)
When I say reliable, I don’t mean “they can fail once in a thousand test runs.” I mean “they must not fail more often than once in a million test runs.” We have around 15k test cases, and they’re run around 70 times a day. That’s a million test cases a day. Even with a literally one in a million chance of an intermittent failure per test case we would still expect to see an intermittent test failure every day.
He means that test failures are not acceptable in their culture, and they have a LOT of tests.
He means that test failures are not acceptable in their culture, and they have a LOT of tests.