The thing humans have, that so far trained machine-learning models seem to lack, is a fairly reliable BS detector. I think the best example of this is an inkblot test. Sure I can look at an inkblot and see fanciful shapes, maybe a chair, maybe two rabbits kissing, whatever. But in the back of my mind, I also know perfectly well that I'm looking at an inkblot; I can entertain all the other classifications, but would never accept them as fact.

The question becomes, how do we give an AI the concept of a BS detector? How can we train it to know, somehow, that the classification it has just made with a high degree of confidence makes no sense? This seems like a very difficult problem to solve.