ChatGPT terms disallow its use in providing legal and medical advice to others

Re:ChatGPT terms disallow its use in providing legal and med... by ChadNauseam on Wednesday November 05, 2025 @09:43PM (#45830792)

So if you set temperature=0 and run the LLM serially (making it deterministic) it would stop hallucinating? I don't think so. I would guess that the nondeterminism issues mentioned in the article are not at all a primary cause of hallucinations.

[ Parent | on HN ]

Re:ChatGPT terms disallow its use in providing legal and med... by joquarky on Wednesday November 05, 2025 @09:55PM (#45830863)

I thought that temperature can never actually be zero or it creates a division problem or something similar.

I'm no ML or math expert, just repeating what I've heard.

[ Parent | on HN ]

Re:ChatGPT terms disallow its use in providing legal and med... by ChadNauseam on Wednesday November 05, 2025 @10:35PM (#45831085)

That's an implementation detail I believe. But what I meant was just greedy decoding (picking the token with the highest logit in the LLM output), which can be implemented very easily

[ Parent | on HN ]

Re:ChatGPT terms disallow its use in providing legal and med... by mbesto on Thursday November 06, 2025 @09:22AM (#45835560)

Did you read the whole article?

"In other words, the primary reason nearly all LLM inference endpoints are nondeterministic is that the load (and thus batch-size) nondeterministically varies! This nondeterminism is not unique to GPUs — LLM inference endpoints served from CPUs or TPUs will also have this source of nondeterminism."

[ Parent | on HN ]