How do we know which sources contain factually right things? What happens when the facts change? It used to be a "fact" that the Sun revolved around the Earth, and that stomach ulcers were caused by stress...

The model can describe the distribution of answers and their confidences. So there will not be one right answer for everything.

Nonsense. There is no reliable way to model confidence on such issues.

That's why repeatable experiments are so important to science. Anybody can independently verify a testable claim.

You shouldn't be so confident without knowing how these things work, there is a absolutely a simple and built-in way to model this... LMMs for example are simply calculating the next word or phrase sequence that is most likely given previous results and modeling information. So they can definitely tell you the combined liklihood that the answer is 'Peru is a cat' vs 'Peru is a country' and provide you the exact statistical likelihood of each.

Actually I am confident exactly because I do know how LMMs work, and your comment fails to address the issue at all. Such models can't tell you anything useful about the probability that a particular statement is accurate.

"So they can definitely tell you the combined liklihood that the answer is 'Peru is a cat' vs 'Peru is a country' and provide you the exact statistical likelihood of each..."

...in the context of the texts that the LLM is built on. Not in the context of the real world, where P('Peru is a country') = 1.0 and P('Peru is a cat') = #cats named Peru / #things in the world (or something).

Most of your world is text. Sure there's a sliver that isn't, but the reality you directly see is a tiny fraction of the reality you know from reports. Come to think of it, I've only ever seen reports of Peru, never actual Peru.

That's how likely they are to occur near each other, not how likely either statement is true. Rude of you to preface your comment the way you did while making this error.

parent post isn't arguing which thing is capital T true (if such a judgement is even universally possible). They are talking about modeling statistical confidence, which is purely an emergent numerical property of data and makes no commentary on objective truth.

Hahaha. Reliable confidence, isn't that a contradiction in terms? If a model makes mistakes but has great confidence estimation why wouldn't it make fewer mistakes in the first place, since it knows when it is wrong.

But if a LM looks up a topic and sees contradictory answers, and none of them is much more reputable, maybe it can still use that information to say it is inconclusive. Knowing a topic is controversial or not present in search engines is useful information. chatGPT would hallucinate, a search + chatGPT solution would refrain from hallucinations. It could also give references.

How is that different with LLMs versus badly-written human generated content? Most clickbait/SEO articles are as poorly researched as they come, and shouldn't be assumed to be accurate anyway.

LLMs don't judge truth. They judge which answer is statistically correct. There is a difference.

They judge the maximum likelihood answer. I take issue with your usage of the word "correct" because it can too easily be confused with "accurate."

The statistically correct answer is not necessarily the true one (if there is such a thing as 'truth'). Many people can believe something to be true, and if I query those people i can calculate which answer is statistically correct. That's the 'wisdom of the crowd'.

Calling a flat earth or a geocentric universe "statistically correct" at past historical points is really inane, don't you think? In doing so, you abuse the notion of what statistics is supposed to represent, which is, generally, a statement of an estimate (and/or distribution), as well as the precision of that statement. Since "correct" is binary, it carries an implied precision of 100%, which renders the notion of "statistically correct" is pretty absurd.

The "wisdom of crowds" is mostly bullshit. It works fine for trivial things like estimating the number of beans in a jar. So what. It completely fails for anything requiring deep expertise.

Seems like this is what you get when you have programmers try to be statisticians, I suppose.

"Statistically correct" gobbledygook that signifies nothing.

Even humans use few relatively simple heuristics to decide what to trust.

One is that objective truth is internally self-consistent. If one AGW denier claims it's the sun, and another AGW denier claims the NASA falsifies the data, and they support each other, then you can judge these are conflicting claims and decrease your trust.

Also, false claims usually focus on attacking competing claims than to come up with a coherent alternative. And they tend to be more vague in specifics (to avoid inconsistency), compare for example vague claims about all scientific institutions faking data vs Exxon files containing detailed reports for executives.