Re:How the Claude Code team thinks about verification
by octoberfranklin on Tuesday November 17, 2026 @07:39PM
(#50540338)
Nothing is ever perfectly hermetic.
Unless your code is 100% single-threaded you'll always have variable timing between threads (influenced by things like caches, etc).
Tests need to be not just hermetic but also deterministic. The problem is that determinism doesn't have a glib "just wrap it in a sandbox bro" way to achieve it.
Re:How the Claude Code team thinks about verification
by FartyMcFarter on Wednesday November 18, 2026 @09:03AM
(#50546461)
> Nothing is ever perfectly hermetic.
Strictly speaking that's true, since there's hardware involved.
But if you introduce reasonable assumptions such as having non-faulty hardware on the vast majority of test runs, most unit tests are in fact perfectly hermetic and deterministic for all practical purposes.
Unit tests are usually single-threaded too, which cuts off a huge source of nondeterminism. Integration tests can quickly get more messy though.
If the tests are hermetic, you should always be able to find the root cause (by bisecting to determine when the test became flaky).
If the tests are not hermetic, it should still be possible to identify the dependency that is making them flaky, and use that as a justification if it still makes sense to delete the test.