News for hackers. Stuff that matters.
Re:Open-weight AI is having its Kubernetes moment. Let's not... by andriy_koval on Saturday July 25, 2026 @08:15PM (#49053229)
It happened that I am working on OSS LLM -> finetuning -> benchmark with 100k tests pipeline, and unless I do some data augmentation, result is 100% deterministic.

I think you likely right, that some parts of stack could induce some marginal float point error, but converged model can mitigate it, and on some principal set of knowledge can give deterministic result with high probability.

Which leads me to believe if you give this task to Anthropic, who has very strong incentive, they will build such benchmark, and then can tell that benchmark gives correct answer with 99.9% probability and it will be enough to drag someone to court.

Re:Open-weight AI is having its Kubernetes moment. Let's not... by nareshshah139 on Sunday July 26, 2026 @01:38AM (#49055072)
The major cause for non-determism is batching.
Re:Open-weight AI is having its Kubernetes moment. Let's not... by antonvs on Sunday July 26, 2026 @12:06PM (#49059510)
If you're running on a single machine with a single GPU, then you may get deterministic results, although it still depends a lot on the details. For example if you're using Pytorch, you need to enable deterministic algorithms and may need to configure some other things as well.

However, running in production at any sort of scale often involves multiple machines and multiple GPUs, and at that point, determinism can be difficult to achieve.

Re:Open-weight AI is having its Kubernetes moment. Let's not... by andriy_koval on Sunday July 26, 2026 @12:31PM (#49059726)
No, I am not running on single GPU, but even then, nothing prevents to use single GPU for models verification.