News for hackers. Stuff that matters.
Re:Non-determinism in GPT-4 is caused by Sparse MoE by sebzim4500 on Friday August 04, 2023 @06:29PM (#37006744)
I would imagine that most people use nonzero temperature, so they won't need to look for any explanation for non-determinism.
Re:Non-determinism in GPT-4 is caused by Sparse MoE by dekhn on Friday August 04, 2023 @06:33PM (#37006788)
Literally the first thing I did when I had llama.cpp working was set the temperature to 0 and repeat queries.

(but that's mainly because I'm a weird old scientist with lots of experience with nondeterminism in software).

Re:Non-determinism in GPT-4 is caused by Sparse MoE by refulgentis on Friday August 04, 2023 @09:49PM (#37008170)
I did too, Kmeans broke me a couple years ago: but, never temperature at 0 with long length, and trusted my instinct instead of actual diffs. This is was the first time I actually diffed