Re:Non-determinism in GPT-4 is caused by Sparse MoE
by eoerl on Sunday August 06, 2023 @01:54AM
(#37019348)
There are flags[1] for that indeed. It feels like half of the people commenting here don't know all that much about the topic they're commenting upon
1: https://pytorch.org/docs/stable/generated/torch.use_determin...
If your code offers an expectation of determinism then it's sloppy to not distinguish where there isn't determinism. There's nothing difficult about writing a comment to the effect of "this function is non-deterministic. For deterministic results, use X".
The code is sloppy if the developers didn't consider determinism and offer nothing to consumers, or if the consumers writing software cannot know where non-determinism is introduced.
If that's somehow insulting then I'd say someone has very thin skin.