I pasted those examples from an earlier, less nuanced discussion; the last example was tongue-in-cheek and I probably shouldn't have pasted it here.

Still: "AI don't know what they don't know" is not universally true and shouldn't be taken at face value. They clearly are able to tell when they're missing knowledge to answer a question. That is distinct from giving the wrong answer because the wrong tokens were sampled for whatever reason.