As we have seen from history, there is not often an absolute truth to questions, only clusters of truths. We want our LLM to be able to perform reasoning, mathematics, and science, but expecting absolute truths in anything outside of those fields is a bit much. Wikipedia often takes a good approach here and strikes this balance well. You can represent the "crazies" and show their reasoning, and then draw attention to how it is commonly refuted. You don't get this if you just omit the "crazies" to begin with.
Edit: What I'm saying is that you can include/represent these adjacent clusters of answers instead of asserting one cluster to be true and allow the correct answer to be represented organically. And guess what? Some people will still disagree for whatever reason they have to disagree. We should be taking all this AI alignment money and putting it into education imo.
An expertise is needed from the reader when presented with false material. That's why so many people think parody news articles are real.
If you have ever gone through any comment thread on the internet about climate change you'll find a large volume of climate deniers citing God, greedy liberals, George Soros, whatever.
The people working with these fictions can always just add more fiction to fill in whatever hole. They aren't bound by reality or credulity.
Actually understanding global climate systems takes an incredible amount of study. You just can't blurt out a few paragraphs of words and get people to stop believing in Jewish weather machine conspiracies or whatever they're thinking.
We would have been done with that by now if this were the case.
Dislodging bullshit is hard. Humanity only started figuring out how a few hundred years ago and we're still fucking it up - such as in the replication crisis
If you're worried that there is more text on the internet saying that the Earth is flat than there is text saying that the Earth is round, then I understand where you are coming from, but then how would ANYONE come to believe that the Earth is round on their own? By learning how to reason.
Edit: My point is that if the LLM can't weed through bullshit properly then how can we expect it to get anything right at all? We can't just feed it a continuously updated list of facts that are true and facts that are false until the end of time.
Because it's not thinking, it's looking at probabilities. If you don't prune branches, you've created a disinformation radiator if that information is sufficiently populous within your corpus to be a pretty-good-bet.
The best analogy I've come up with is "it's like if you had an objective reality based on Wikipedia edit wars."
All that being said, who is to say that humans aren't just looking at probabilities when they think?
Some scams, like MLM, encourage massive volumes of this bullshit to be created and disseminated.
The democracy of information does not lead to an increased accuracy based on whatever information spreads the most. The internet is a multi-decade long global proof of this. There's a reason that classrooms have 1 teacher and say 24 students - only 4% of that room is to be trusted with the material.
Humans will argue the right answer until their last days. It's frustrating how on-the-fence chatgpt can be. It's pretty interesting too, because in a professional environment one of the most important things you need to do is have an opinion and take a position otherwise you can't execute.
To English learners, that might be very difficult. I'm learning a new language now and I feel the pain. The textbook and instructor, in this case, is way more correct than the collective opinion of my fellow students.
The vast majority of things actually follow this pattern instead. Knowing what a programming function would output given a set of inputs isn't some gray area world of opinion.
When things are unknown, such as say Covid origin, again there's people way beyond my pay-grade doing investigative work here. Instagram comment threads and tiktok opinions are actually not as valuable as peer reviewed scholarship on the topic EVEN IF there's more instagram comments saying one thing than papers in the Lancet saying another. The Instagram commenters don't become correct by volume here.
It's not some giant equal playing field.
As a linguistic descriptivist, hahahahahahahaha.
Linguistic descriptivism just means that, rather than hold up an ideal of a language and prescribe variants as wrong or right, linguists should simply describe the language based on its use. That doesn't mean that the language (or variants of it) has no grammatical rules. It just means that, rather than holding up some prestige variety as "the language," and unprestigious ones as "uneducated errors" that shouldn't be studied, that you study all of them and determine how they work and how they're developing.
No matter where you go, people speak languages with a limited set of phones, which are mapped onto morphemes, which combine by particular rules to form words, which themselves form larger groups, like phrases and sentences (often the line between these things isn't clear cut). But languages all have rules of their own.
To imply that descriptivism means languages have no rules would be like saying that physics has no laws, because a physicist makes empirical observations instead of just deciding whatever the laws of physics ought to be.
Even at the lowest level of phonemes to morphemes, accents introduce a huge amount of variety, and it doesn't really get more orderly as you increase the level of abstraction.
Or did I break some rules there?
I'm totally onboard with that.
At the same time, when talking about the physical world, most things are not like that. Humans for instance, get around by moving limbs called feet and not flapping feathered wings like a bird. This is the level of obvious I'm talking about.
The secondary point was that many things are this level of obvious and also specialized. For example: how does the animal cacomantis castaneiventris get around? Unless you have some kind of specialized knowledge you wouldn't know that's a type of bird.
The internet is awash in unspecialized people opining on specialized knowledge that is only controversial to the untrained eye in the same way that students don't get As on every exam they take. You need to actively work around people talking out of their bounds if you want your training data to be accurate
Using everything would be like taking a bunch of students final exams and then claiming the most common answers are the correct ones.
This isn't how expertise and accuracy works. Most things worth doing are not only genuinely hard and complicated but something that only a minority subset of accomplished people can do consistently well in.
Listening to everybody and incorporating their thoughts is only going to lead to wrong answers.
Being selective is the key here unless you genuinely want say, answers about to space to involve aliens and UFOs - because way more people believe in that then there are qualified PhD astrophysicists in the world.
Similarly, way more people believe in vaccine conspiracy theories then there are people with significant viral epidemiology backgrounds.
This pattern is true in every field.