News for hackers. Stuff that matters.
Re:Ask HN: Why are LLM's made intentionally non-deterministic? by Cr8 on Tuesday January 06, 2026 @02:41AM (#46509705)
unfortunately disabling temperature / switching to greedy sampling doesn't necessarily make most LLM inference engines _fully_ deterministic as parallelism and batching can result in floating point error accumulating differently from run to run - it's possible to make them deterministic but does come with a perf hit

some providers _do_ let you set the temperature, including to "zero", but most will not take the perf hit to offer true determinism

There are no comments on this comment yet.