The most common mistake you'll get here is people running tests, then making a small change later, then having tests fail unexpectedly. Even accidentally. This has happened to me before with regular CI, and often hit me during an automerge I fully expected to pass.