This isn't really true when you control the stack, no? If you have all of your parameters set to be reproducible (e.g. temp 0, same seed), the output should be the same as long as everything further down the stack is the same, no?

That's not a usable workaround. In most cases it doesn't actually produce full determinism[1].

And even if it did, a certain degree of non-determinism is actually desirable. The most probable tokens might not be correct, and randomness is partly responsible for what humans interpret as "creativity". Even hallucinations are desirable in some applications (art, entertainment, etc.).

[1]: https://medium.com/google-cloud/is-a-zero-temperature-determ...

Is that critical? Doesn't it just need to be better than the alternative? Unless it's a safety-critical system.