That's why repeatable experiments are so important to science. Anybody can independently verify a testable claim.

You shouldn't be so confident without knowing how these things work, there is a absolutely a simple and built-in way to model this... LMMs for example are simply calculating the next word or phrase sequence that is most likely given previous results and modeling information. So they can definitely tell you the combined liklihood that the answer is 'Peru is a cat' vs 'Peru is a country' and provide you the exact statistical likelihood of each.

Actually I am confident exactly because I do know how LMMs work, and your comment fails to address the issue at all. Such models can't tell you anything useful about the probability that a particular statement is accurate.

"So they can definitely tell you the combined liklihood that the answer is 'Peru is a cat' vs 'Peru is a country' and provide you the exact statistical likelihood of each..."

...in the context of the texts that the LLM is built on. Not in the context of the real world, where P('Peru is a country') = 1.0 and P('Peru is a cat') = #cats named Peru / #things in the world (or something).

Most of your world is text. Sure there's a sliver that isn't, but the reality you directly see is a tiny fraction of the reality you know from reports. Come to think of it, I've only ever seen reports of Peru, never actual Peru.

That's how likely they are to occur near each other, not how likely either statement is true. Rude of you to preface your comment the way you did while making this error.

parent post isn't arguing which thing is capital T true (if such a judgement is even universally possible). They are talking about modeling statistical confidence, which is purely an emergent numerical property of data and makes no commentary on objective truth.

Hahaha. Reliable confidence, isn't that a contradiction in terms? If a model makes mistakes but has great confidence estimation why wouldn't it make fewer mistakes in the first place, since it knows when it is wrong.

But if a LM looks up a topic and sees contradictory answers, and none of them is much more reputable, maybe it can still use that information to say it is inconclusive. Knowing a topic is controversial or not present in search engines is useful information. chatGPT would hallucinate, a search + chatGPT solution would refrain from hallucinations. It could also give references.