Re:AI Isn't Magic, It's Maths
by xigoi on Saturday June 14, 2025 @02:32AM
(#44274635)
Last time I checked, modern LLMs could give you a different answer for the same sequence of prompts each time. Did that change?
[ Parent
| on HN ]
Re:AI Isn't Magic, It's Maths
by perching_aix on Saturday June 14, 2025 @02:54AM
(#44274695)
They can, but won't necessarily. If you use a managed service, they likely will, due to batched inference. Otherwise, it's simply a matter of configuring the seed to a fixed value and the temperature to 0.
At least that's what I did, and then as long as the prompts were exactly the same, the responses remained exactly the same too. Tested with a quantized gemma3 using ollama, I'd say that's modern enough (barely a month or so old). Maybe lowering the temp is not even necessary as long as you keep the seed stickied, didn't test that.
Re:AI Isn't Magic, It's Maths
by Workaccount2 on Saturday June 14, 2025 @08:26AM
(#44276000)
Just a note for everyone, people do these fixed prompts with fixed seed and 0-temp tests to track model versions. A cool application of the technique
Re:AI Isn't Magic, It's Maths
by perching_aix on Saturday June 14, 2025 @10:29AM
(#44276657)
Small update: now that I brought it up, I figured I should just test it. And indeed, as long as the seed is pinned, the replies are bit-for-bit identical run to run (note that the conversation context was cleared before each attempt, of course).
So even setting the temp to 0 is not actually needed. This is handy in case somebody makes the claim that the randomness (nonzero temp parameter) makes a model perform better.
The devil really is in the "insignificant for humans but significant for the model" details, basically. Not in the computational determinism.
A random number generator also gives deterministic results when you fix the seed. Does that make it not a random number generator?
Re:AI Isn't Magic, It's Maths
by perching_aix on Saturday June 14, 2025 @11:56AM
(#44277129)
Yes, it does. They're called pseudo-random number generators (PRNGs) in literature for that very reason.