How outdated information hides in LLM token generation probabilities

Re:How outdated information hides in LLM token generation pr... by sd9 on Sunday January 12, 2025 @07:29AM (#42673099)

I think it would be more vibes based - commonly occurring things would be reinforced more in the weights. Rather than it explicitly counting the number of occurrences.

[ Parent | on HN ]

Re:How outdated information hides in LLM token generation pr... by diggan on Sunday January 12, 2025 @09:03AM (#42673594)

So the probabilities would be skewed towards something, but unless the model could somehow count/infer its own weights, I don't see how it could "introspect" to see if something is more common than something else.

[ Parent | on HN ]