News for hackers. Stuff that matters.
Re:AI Isn't Magic, It's Maths by perching_aix on Saturday June 14, 2025 @02:54AM (#44274695)
They can, but won't necessarily. If you use a managed service, they likely will, due to batched inference. Otherwise, it's simply a matter of configuring the seed to a fixed value and the temperature to 0.

At least that's what I did, and then as long as the prompts were exactly the same, the responses remained exactly the same too. Tested with a quantized gemma3 using ollama, I'd say that's modern enough (barely a month or so old). Maybe lowering the temp is not even necessary as long as you keep the seed stickied, didn't test that.

Re:AI Isn't Magic, It's Maths by Workaccount2 on Saturday June 14, 2025 @08:26AM (#44276000)
Just a note for everyone, people do these fixed prompts with fixed seed and 0-temp tests to track model versions. A cool application of the technique
Re:AI Isn't Magic, It's Maths by perching_aix on Saturday June 14, 2025 @10:29AM (#44276657)
Small update: now that I brought it up, I figured I should just test it. And indeed, as long as the seed is pinned, the replies are bit-for-bit identical run to run (note that the conversation context was cleared before each attempt, of course).

So even setting the temp to 0 is not actually needed. This is handy in case somebody makes the claim that the randomness (nonzero temp parameter) makes a model perform better.

The devil really is in the "insignificant for humans but significant for the model" details, basically. Not in the computational determinism.

Re:AI Isn't Magic, It's Maths by xigoi on Saturday June 14, 2025 @11:39AM (#44277026)
A random number generator also gives deterministic results when you fix the seed. Does that make it not a random number generator?
Re:AI Isn't Magic, It's Maths by perching_aix on Saturday June 14, 2025 @11:56AM (#44277129)
Yes, it does. They're called pseudo-random number generators (PRNGs) in literature for that very reason.