News for hackers. Stuff that matters.
Re:Advanced NLP with SpaCy by dartos on Saturday September 09, 2023 @06:01PM (#37450727)
Only the training is non deterministic. As long as you use the same seed for inference, it’ll give you the same answer.

ChatGPT changes this seed under the hood with each request

Re:Advanced NLP with SpaCy by squeaky-clean on Sunday September 10, 2023 @04:18AM (#37453894)
Even if you keep the same seed, it is non-deterministic.

https://ai.plainenglish.io/gpt-4-is-non-deterministic-af2028...

Re:Advanced NLP with SpaCy by dartos on Tuesday September 12, 2023 @07:36PM (#37490159)
That link is dead. Either way, you can’t interact with gpt4 without openai’s api, so you don’t know what they’re doing in between your request and inference.

But LLMs are deterministic when all the parameters are the same between inferences