On the other hand, if you're talking to someone you've never met over the internet and they're telling you the shed in their back yard is painted red, what confidence do you have in that? How do you even know you're talking to a person?
LLMs are trained on large amounts of text that's just been scraped off the internet. Some of it may be high quality (if you want to call Wikipedia high quality) but much of it (Reddit) is not. When you ask an LLM chatbot a question, how do you know you're getting the Wikipedia answer and not the Reddit one? You don't. It's all mashed up. Until LLMs are able to reliably provide citations for their answers, they ought to be relegated to the epistemic dustbin.
Yet I see high schoolers relying on them all the time. They will ask ChatGPT a question and write down its answer verbatim. No second-guessing, no fact-checking, nothing. Blind trust. It's appalling!
My professors have even told me they've had students hand in papers full of citations. Turns out, all the citations were completely fabricated by ChatGPT. One professor even said the paper a student submitted cited HIM, the professor receiving the assignment, for a completely made up paper that never existed. When confronted, the students admitted all of the papers were ChatGPT-generated nonsense.
Even worse than the Reddit problem above is the problem of LLM-generated text proliferating throughout the internet. The next generation of LLMs are going to be polluted by that nonsense. Garbage-in, garbage-out. They'll be no more trustworthy than a psychiatric patient rambling to themselves in a padded cell.
I think this is a good idea if LLMs justification was 0. It's not, and it's not clear what it is. That people talk to a storyteller thinking that they talk to a scholar is on the people, or on whoever misrepresented the interlocutor, not on the interlocutor.
It's marvelous already that we can't rule out whatever nonzero amount of justification in the case of LLMs.
We absolutely can rule it out. The justification is zero. The LLM has not and cannot verify any of the factual claims in its text. It can't even tell a claim apart from a non-claim. In legal parlance, the entire training set of an LLM is hearsay. LLMs do not produce any first party information. It's 100% third party.
Asking an LLM about the physical world is not unlike asking a historian about the past. All information they have past a certain point is 3rd party, mediated by tens/hundreds of years. Can historians say anything justifiable about the past? So can LLMs about the physical world.
If you want to rule out historians being able to justify their knowledge, I'd like to hear that.
At the same time, if you ask historians about (say) the Peloponnesian War they will tell you about the account of Thucydides. What they won’t say is that they know about it personally.
That’s a mistake a lot of non-scientists make about science. They’ll hear scientists say things like “we know this” or “we know that” and think that they are committing to some privileged knowledge of the world. They aren’t. They’re using the words “we know X” as shorthand for “our best theory about X.” In fact, many scientists identify as scientific anti-realists [1] which means they don’t believe science has any true facts about the nature of reality at all and that the only job of science is to make better predictions about the outcome of experiments.
Of course, if you take anti-realism to its logical conclusion then you have to admit that fossils are only evidence that you may find more fossils in the future, and that you have no basis to talk about the existence of dinosaurs.
Personally, I think our actual knowledge of things directly outside our personal experience is very limited. But then I want to encourage people not just to believe everything they read. People should be exploring and trying their own science experiments and trying to build things to understand how the world works.
[1] https://plato.stanford.edu/entries/scientific-realism/#AntiF...
> Historians have access to more than just texts about the past. They have access to physical evidence. Radiometric dating, artifacts, geographic data about where things were found.
Correct. That's still all indirect, "third-party" evidence. Does that make answers based on it impossible to justify?
To make this analogy even closer to LLMs, let's take historian scholars (linguists? undergrads?) who have nothing to study but books. No experiments, no carbon dating, just texts. Is their entire knowledge about the past impossible to justify?
Historians who only have books to go on have frequently said the accounts they're using should be treated as only that: accounts, not facts about what happened.
For example, even a figure as popular as Socrates is "known" only through the accounts of Plato and Xenophon. What do we know about him? The two accounts are in conflict about many details of his personality. So we can attribute very little in the writings of Plato to actual opinions of Socrates. Scholars treat these books as the opinions of Plato, expressed through dialogues of Socrates.
Am I misrepresenting your position?
But when I present an analogous situation, except with historians playing the main role, you seem to accept the justifiability of their statements on a spectrum, rather than rejecting it based on the evidence being textual and indirect.
This is the crux of the discussion, without explaining which we won't come to a mutual understanding (much less acceptance).
However, this does not happen in real life. Text does not fall from the heavens into the minds of historians. It comes in books, manuscripts, stone tablets, etc. Unlike digital text (which is the only thing LLMs have access to), physical texts have physical properties which can be examined experimentally. They can be subject to radiometric dating, they typically are found in a particular place (which is usually recorded), and they have version and copyright information which enables them to be traced back to the original. In the case where the original manuscript is lost (such as Thucydides’ history of the Peloponnesian War), we have to rely on copied manuscripts by other scribes, often made centuries later.
All of this is to say that it can be very difficult to trace a book back to its original author, especially without the original manuscript, but the evidence historians use to piece all this together is generally not in the text itself. It’s physical evidence. Exactly the thing that LLMs have no access to.
Let me protest against this broader point.
If I'm a historian, do I have to find the physical text in the original place myself, or can this information be supplied to me in the form of text? Do I have to actually see the radiocarbon dating machine work myself, or is it enough that the operator sends me an email with the results? What you describe doesn't really strike me as evidence that cannot be supplied through text - you mentioned that it's recorded yourself. It's still text about the past, maybe even with an extra level of indirection (it describes the ~present which has some information about the past).
It follows that some of LLMs statements about historical artifacts should be justifiable - those coming from the written word that our hypothetical historian would study. Is that not correct?
The other problem with discounting LLMs based on the form of sensory input that's available to them is that there is no qualitative comparison of validity of different senses. Sure, getting more visual input counts as more evidence about the world (bar hallucinations). But can part of it be substituted with text without losing the quantitative benefit? Can all of it be substituted with text? You seem to answer "no", but I think it's an oversimplification. Not that I have a definite answer.
The one thing that LLMs cannot currently do is design and perform experiments - or if they can, then they can't integrate the results. I'd look for ways LLMs cannot reach some level of justification for beliefs there. But I would not presume that this level is automatically lower than what humans can reach. I have not compared the size of a human training set to that of an LLM.