> These systems are constantly being compared to perfectly rational humans, and getting the wrong answer seems to be taken as being a fault. But in a world of flat-earthers and truthers, why is the bar for general intelligence being set so high for LLMs?

The input of this was a collection of data produced by humans. The by default behavior (if advanced) would presume to be human-like, and yet everyone seems to want to set the bar to be an all knowing oracle.

It's such a fundamental mistake people keep making.

There are a whole lot of humans out there. Not to mention there are 5 year-old humans, 3 year old humans. We seem unable to imagine for example a human child before developing theory of mind, but with an adult's vocabulary is still extremely human like. LLMs may not develop in conjunction with how humans develop, they may get some skills earlier some later. They may never get it from LLMs. But we need to get better at judging them for what they are capable, and not simply dismissing them because they aren't a perfect oracle nor a Phd grad.