News for hackers. Stuff that matters.
Re:Grok: Searching X for "From:Elonmusk (Israel or Palestine... by lgessler on Thursday July 10, 2025 @09:10PM (#44527453)
In my (poor) understanding, this can depend on hardware details. What are you running your models on? I haven't paid close attention to this with LLMs, but I've tried very hard to get non-deterministic behavior out of my training runs for other kinds of transformer models and was never able to on my 2080, 4090, or an A100. PyTorch docs have a note saying that in general it's impossible: https://docs.pytorch.org/docs/stable/notes/randomness.html

Inference on a generic LLM may not be subject to these non-determinisms even on a GPU though, idk

Re:Grok: Searching X for "From:Elonmusk (Israel or Palestine... by msgodel on Friday July 11, 2025 @11:39AM (#44533405)
Ah. I've typically avoided CUDA except for a couple of really big jobs so I haven't noticed this.