News for hackers. Stuff that matters.
Re:Advanced NLP with SpaCy by squeaky-clean on Sunday September 10, 2023 @04:17AM (#37453890)
ChatGPT 4 is inherently non-deterministic. Even across the same exact model version with temperature set to 0.

https://ai.plainenglish.io/gpt-4-is-non-deterministic-af2028...

Re:Advanced NLP with SpaCy by yorwba on Sunday September 10, 2023 @08:55AM (#37455409)
That's what I meant by "if you're using some API you have no control over, anything could happen." OpenAI could offer access to a fully deterministic version at a higher price point, but they choose not to and there's nothing you can do about that.

FYI, your source is medium-walled blog spam of https://152334h.github.io/blog/non-determinism-in-gpt-4/

Re:Advanced NLP with SpaCy by squeaky-clean on Monday September 11, 2023 @02:06AM (#37463472)
If you'd read the source you linked you'd see that no, the system is inherently non-deterministic. They cannot sell you a deterministic version of GPT-4. There is no such thing. They could sell you a deterministic version of some other inferior model.
Re:Advanced NLP with SpaCy by yorwba on Monday September 11, 2023 @07:15AM (#37465542)
From the source, which I read:

As a consequence, the model is no longer deterministic at the sequence-level, but only at the batch-level

They could sell you a version where you shove in a whole batch, pay $batch_size as much, and always get the same result for the same batch.