Maybe this insight is why I feel hallucinations are much rarer in the last 12 months on top models. Are they being detected before they get sent out.
Hallucinations may seem rarer for a few reasons. First, models are more accurate with certain prompts. Second, models are more convincing when they do hallucinate. They may get an overall idea, but hallucinate the details. Hallucinations are still a major problem and are fundamental to the way modern LLMs work.
You can't look at a single token distribution, though. There are many legitimate high-confidence, high-accuracy cases in which many tokens could come next. For example, the first token of a paragraph. You need to look at pools of entropies over segments of the output or the whole output sequence.
Although there's a correlation between uncertainty and hallucinations or inaccuracies, there's no guarantee. This is a challenging area that we're monitoring the latest literature for and contributing where we can.
There is interesting research in the correlation of entropy with accuracy and hallucinations: - https://www.nature.com/articles/s41586-024-07421-0 - https://arxiv.org/abs/2405.19648 - https://arxiv.org/abs/2509.04492 (when only a small number of probabilities are available, which is something we frequently deal with) - https://arxiv.org/abs/2603.18940 - tons more, happy to chat about if interested