My understanding is the bugginess is due to* unintended* nondeterminism, in other words, things like a race condition where two threads write some result to the same memory address, or singularities/epsilon error in floating point calculations leading to diverging results.

Make no bones about it, these are programming faults. There's no reason why distributed, long-running models can't produce convergent results with a high degree of determinism given the input state. But this takes some amount of care and attention.

> So you might want to change the seed on every run to protect yourself from such nasty phantom effects.

That's a perfect example of what I mean where a seed is actually worse. If you know you can't control determinism, then you might as well go for the opposite: ensure your randomness is high quality enough such that it approximates a perfect uniform distribution. Adding a seed here means you are more likely to capture the true distribution of the output.