News for hackers. Stuff that matters.
Re:AI Hallucinations: Why Large Language Models Make Things ... by nomel on Thursday December 05, 2024 @04:34AM (#42326506)
They mean the non-normalized probabilities for each tokens is available. Many API give access to the top-n. You can color the text based on it, or include it in your pipelines, like trigger looking externally, or inject claims of uncertainty (the same things I do). It's not remotely guaranteed, but it's some low hanging fruit that can sometimes be useful.

One of these days, someone will figure out how to include that in the training/inference loop. It's probably important for communication and reasoning, considering a similar concept happens in my head (some sort of sparsity detection).

There are no comments on this comment yet.