Re:Non-determinism in GPT-4 is caused by Sparse MoE
by derwiki on Friday August 04, 2023 @06:16PM
(#37006596)
GPT4 web chat for two hours a day? I buy that. Using the API repeatedly for the same inputs, eg developing a program, and the non-determinism is hard to miss.
[ Parent
| on HN ]
Re:Non-determinism in GPT-4 is caused by Sparse MoE
by sebzim4500 on Friday August 04, 2023 @06:29PM
(#37006744)
I would imagine that most people use nonzero temperature, so they won't need to look for any explanation for non-determinism.
Re:Non-determinism in GPT-4 is caused by Sparse MoE
by dekhn on Friday August 04, 2023 @06:33PM
(#37006788)
Literally the first thing I did when I had llama.cpp working was set the temperature to 0 and repeat queries.
(but that's mainly because I'm a weird old scientist with lots of experience with nondeterminism in software).
Re:Non-determinism in GPT-4 is caused by Sparse MoE
by refulgentis on Friday August 04, 2023 @09:49PM
(#37008170)
I did too, Kmeans broke me a couple years ago: but, never temperature at 0 with long length, and trusted my instinct instead of actual diffs. This is was the first time I actually diffed
Re:Non-determinism in GPT-4 is caused by Sparse MoE
by phillipcarter on Friday August 04, 2023 @07:40PM
(#37007292)
Yeah, it's one of the first things you notice when trying to do some kind of "feed GPT some data and get it to produce a novel answer to a question" task with the API.
Re:Non-determinism in GPT-4 is caused by Sparse MoE
by refulgentis on Friday August 04, 2023 @09:48PM
(#37008165)
No, because if you wanted a novel answer, why would you set 0 temperature? ;)