Re:Wronger Than Wrong
by danielvf on Monday April 03, 2023 @09:16AM
(#35422977)
An LLM like ChatGPT ends up with a bunch of probabilities for the next word-ish. A random number generator then picks from them, weighted by probability, and then the process loops. So while the weights are static, the actual execution is very much non-deterministic.
[ Parent
| on HN ]
Do they use an actual physical source of randomness for every random number? Or is it a prng? Because in that case it is fully deterministic.
And consider the case that in fact they could use actual random numbers. Would it make a difference?
While I can't say whether ChatGPT or others are actually doing this, it's entirely possible to use a PRNG in that spot with no loss of effectiveness. PRNGs are specifically designed to create the appearance of randomness out of a process that is in fact wholly deterministic.
Just because we cannot determine something doesn't mean it's non-deterministic. The question of the existence of free will is very similar – if not equivalent – to the existence of real randomness.
I.e. a dice roll may be deterministic if you just know the right set of complicated equations for gravity, air resistance, turbulence etc.
> the actual execution is very much non-deterministic.
Only as a default. Temperature can be turned to 0 and other token selection settings changed from the defaults to create deterministic output from a given prompt.