Agreed.

I wonder what that would look like?

Something like: if every best next token has a confidence below some threshold for some length of context then the model can confidently assume ignorance?

I would love to read the training corpus required to achieve this without metamodeling.

That would likely be a utopian forum community devoid of ego and gaslighting and filled with humble people with a thirst for knowledge who clearly communicate the limits of their current experience.

Every thread would be a few gems of info sinking in an ocean of "I don't know."

It'd be hard if not impossible to include what people never write into the corpus.