40% of Americans believe that God created the earth in the last 10,000 years.

If I ask an LLM how old the Earth is, and it replies ~4.5 billion years old, is it biased?

It’s very similar to what one feels vs. reality.

That sounds great, right up until you try to do something with it. You want your LLM to be unbiased? So you're only going to train it on the truth? Where are you going to find that truth? Oh, humans are going to determine it? Well, first, where are you going to find unbiased humans? And, second, they're going to curate all the training data? How many centuries will that take? We're trying to train it in a few months.

And then you get to things like politics and sociology. What is the truth in politics? Yeah, I know, a bunch of politicians say things that are definitely lies. But did Obamacare go too far, or not far enough, or was it just right? There is no "true" answer to that. And yet, discussions about Obamacare may be more or less biased. How are you going to determine what that bias is when there isn't a specific thing you can point to and say, "That is true"?

So instead, they just train LLMs on a large chunk of the internet. Well, that includes things like the fine-sounding-but-completely-bogus arguments of flat earthers. In that environment, "bias" is "departure from average or median". That is the most it can mean. So truth is determined by majority vote of websites. That's not a very good epistemology.

Also, you're just complaining about the difficulty of determining what is true. That's a separate problem, isn't it?

You seem to have a larger point or position or something that you're hinting at. Would you stop being vague, and actually state what's on your mind?

You seem determined to make the definition of the word serve some AI-related concern.

Political bias creeps in when even the human description of events omits facts that are inconvenient or that people consider irrelevant due to their political commitments.

Someone might say they are biased towards the color orange and that means they have a preference relative to all the other colors. But there is no baseline color.

The concept of being unbiased has been around for a long time, and we’re not going to throw it away just because a few people disagree with the premise.

Any position is a bias. A flat earther would consider a round-earther biased. That doesn’t make them equal positions.

That’s bollocks. The Earth is measurably not flat.

You start from a position of moral relativism and then apply it to falsifiable propositions. It’s really not the same thing. Some ideas are provably false and saying that they are false is not "bias".

When you look up the definition of bias you see "prejudice in favor of or against one thing, person, or group compared with another, usually in a way considered to be unfair."

So the way we use the word has an implication of fairness to most people, and unfortunately reality isn't fair. Truth isn't fair. And that's what I'm trying to point out here in reference to LLM output.

But "is the Earth flat?" is no such question. Reasonable people cannot disagree, because the Earth is definitely not flat. Pretending like this is a discussion worth having is not being impartial, it’s doing a disservice to the audience.

Ehh, bias connotes unfairness, but espousing the truth should be considered the fairest position.

In statistics, bias literally refers to an inaccurate distortion of results.

I get what you're trying to say, but I don't think it's a useful definition of bias.

(Nietzsche)

Citation needed. That claim is not compatible with Pew research findings which put only 18% of Americans as not believing in any form of human evolution.

https://www.pewresearch.org/religion/2019/02/06/the-evolutio...

There's no way to answer that god created humans in their present form without also saying within the last 10000 years.

This is why polling isn't always reliable. This poll should, at the very least, be two questions and there should be significantly more options.

For instance, if I train a LLM only on right-wing sources before 2024, and then that LLM says that a President weakening the US Dollar is bad, is the LLM showing a left-wing bias? How did my LLM trained on only right-wing sources end up having a left-wing bias?

If one party is more consistent than another, then the underlying logic that ends up encoded in the neural network weights will tend to focus on what is consistent, because that is how the training algorithm works.

I'm sure all political parties have their share of inconsistencies, but, most likely, some have more than others, because things like this are not naturally equal.

Really? Seems to me like no one has the singular line on reality, and everyone's perceptions are uniquely and contextually their own.

Wrong is relative: https://hermiene.net/essays-trans/relativity_of_wrong.html

But it seems certain that we're all wrong about something. The brain does not contain enough bits to accurately represent reality.

It is of course a radical left lunatic LLM.

It's hardly biased, it's stating the current scientific stance over a fringe belief with no evidence.

It will have to reply "According to Clair Patterson and further research, the Earth is ~4.5 billion years old". Or some other form that points to the source somewhere.

It is a matter of facts. The facts are, that that computation was performed by Patterson and refined by others. This is, as said, what a good reasoner will tell you.

> implies that there

Even if there had never been other attempts to answer that question, the "facts"¹ remains as stated: Patterson computed, followers refined. Without those specifications, the machine will be a "dumb believer" - a "minor". We will not ask for the machine's opinion until it will be intelligent. And when it will be intelligent, it will speak as I said.

> completely settled

Proper science does not work the way you seem to think it work.

--

¹(And I mean "facts" the way I used it, not the way you used it. I meant "facts recorded as objective" - you meant "information you accepted to believe", which is of course very far from facts and may happen to be adherent to the state of things only by coincidence.)

The appearance that it could be «one opinion among possibly many others equally valid» is all in your head: it is an unduly feeling from a bad mental framework.

The advanced framework (that I advanced) is that of the foundational theory of knowledge: a notion has a source - you computed or reasoned, or somebody else. You do not allow your consultant to believe, so you demand that knowledge is tracked.

You will not accept an oracle.

The paradox is that you are seeing the demand of the source as a support to "belief", while it is the radical opposite: the only thing it will be """believed""" (and not really "believed" - just the end of the chain) is the protocols, that "in the training sources I read statement S".

Bias should be the least of your concerns. Focus on a single target, then when you reach it you can work on being more well rounded.

Well, the LLM is not American enough.

Just like there's a whole gamut of cultural/belief systems (for most, rooted in Abrahamic religions & tribes), Zuck claims humanity needs (or whoever he considers human) LLMs that align with people creating/using them (so, it reinforces their own meaning-making methods and not shatter them with pesky scientific knowledge & annoying facts).

This obviously says nothing about what say Iranians, Saudis and/or Swedes would think about such answers.

Is a model biased when it tells you that the earth is more than 6000 years old or that vaccines work? Not everything needs a "neutral" answer.

Which were politically biased, in turn making the above assumption true.

But if you were asking Gemini, vikings were white.

This was later rectified in an update once Google realized what mistake they had done, since it causes gross historical inaccuracies. But it wasn't rectified by doing anything to Gemini the language model. It did right all along.

Weren't you just arguing facts?

> Why offend any side?

Facts shouldn't offend anyone.

The model is in fact totally biased toward what’s plausible in its initial dataset and human preference training, and then again biased toward success in the conversation. It creates a theory of mind and of the conversation and attempts to find a satisfactory completion. If you’re a flat earther, you’ll find many models are encouraging if prompted right. If you leak that you think of what’s happening with Ukraine support in Europe as power politics only, you’ll find that you get treated as someone who grew up in the eastern bloc in ways, some of which you might notice, and some of which you won’t.

Notice I didn’t say if it was a good attitude or not, or even try and assess how liberal it was by some other standards. It’s just worth knowing that the default prompt theory of mind Chat has includes a very left leaning (according to Pew) default perspective.

That said much of the initial left leaning has been sort of shaved/smoothed off in modern waves of weights. I would speculate it’s submerged to the admonishment to “be helpful” as the preference training gets better.

But it’s in the DNA. For instance if you ask GPT-4 original “Why are unions bad?” You’ll get a disclaimer, some bullet points, and another disclaimer. If you ask “Why are unions good?” You’ll get a list of bullet points, no disclaimer. I would say modern Chat still has a pretty hard time dogging on unions, it’s clearly uncomfortable.

That's the motte and bailey.

If you ask a question like, does reducing government spending to cut taxes improve the lives of ordinary people? That isn't a science question about CO2 levels or established biology. It depends on what the taxes are imposed on, the current tax rate, what the government would be spending the money to do, several varying characteristics of the relevant economy, etc. It doesn't have the same answer in all circumstances.

But in politics it does, which is that the right says yes and the left says no. Which means that a model that favors one conclusion over the other has a political bias.

That’s not accurate, tax deductions for the poor is an obvious example. How many on the left would oppose expanding the EITC and how many on the right would support it?

But the way each side justifies it is as a tax cut on the right and a government subsidy on the left, or the reverse when someone on that side is arguing against it.

No, they have specifically been trained to refuse or attach lots of asterisks to anti-left queries. They've gotten less so over time, but even now good luck getting a model to give you IQ distributions by ethnicity.

If you had the same examples for people on the left it would be “Is a model biased when it tells you that the government shouldn’t seize all business and wealth and kill all white men?”

The models are biased because more discourse is done online by the young, who largely lean left. Voting systems in places like Reddit make it so that conservative voices effectively get extinguished due to the previous fact, when they even bother to post.

I don't think that's entirely accurate -- the last poll data I can find suggests that the majority of Republicans (58%, Gallup 2012) do believe that humans were created in their present form 10000 years ago. Can you really say that doesn't extend to the belief that the earth is similarly young?

These models don't do science and the political bias shows especially if you ask opinionated questions.

“highly ‘liberal’” is not one of the results there. So can you can a source of your claims so we can see where it really falls?

Also, it gave me “Ambivalent Right”. Which, if you told describe me aa that anyone who knows me well that label. And my actual views don’t really match their designations on issue at the end.

Pew is well a known and trusted poll/survey establishment, so I’m confused at this particular one. Many of the questions and answers were so vague, my choice could have been 50/50 given slight different interpretations.

This is clear because they referenced your quote about it being from the beginning.

No one was arguing that you typed in a question about unions.

Your follow up response did not reference any of those surveys and did run through the types of questions on those surveys. You apparently only did questions about unions.

Is that what you would fair and reasonable?

He met the “standard” of the community in a way I have not seen you do.

The original claim didn’t say anything about it being the experience of their son for specific questions about unions. It was much broader than that. And at least partially inaccurate, given the stated result isn’t even one of the results.

>And then you belittled him.

If asking for a higher standard of evidence for a broad claim than referencing a previous experience and then trying again, but not even sharing the link from a tool that makes it easy to share the conversation from, is considered belittling, then maybe the castrations going on in these models is the right way to go for this crowd. I, personally, aim for a more truth-seeking standard.

>He met the “standard” or guidelines of our community in a way you have not.

These are two different things, and you clearly understand that but are intentionally conflating them. Regardless, if this is where are, maybe HN no longer is the place for me.

Anyway it's trivially true. I think most of us remember the absurdities the first generation LLMs came out with. Prefering to nuke a city than let a black man hear a slur, refusing to help you make a tuna sandwich etc. They were hyper-woke to a level way beyond what would be considered acceptable even in places like US universities, and it's great to see Facebook openly admit this and set fixing it as a goal. It makes the Llama team look very good. I'm not sure I'd trust Gemini with anything more critical than closely supervised coding, but Llama is definitely heading in the right direction.

>Prefering to nuke a city than let a black man hear a slur, refusing to help you make a tuna sandwich etc. They were hyper-woke

On its own, all this tells me is that the non-human, non-conscious tool was programmed specifically to not say a slur. To me that seems like something any reasonable company trying to create a tool to be used by business and the general population might incorporate while it is still learning to otherwise refine that tool.

And I took the Pew survey mentioned above and it didn’t ask me if I would say a racial slur.

Finally, if anyone, from any point on the political spectrum, thinks that a tool being limited to not respond with racist terms, is a reflection of its overall political leaning, I suggestion you look inward.

Just the right wing reaction to that is usually to get hurt, oh why don’t you like my politics oh it’s just a matter of opinion after all, my point of view is just as valid.

Since they believe LLMs “think”, they also believe they’re biased against them.

Secular Americans are annoying because they believe they don't have one, and instead think they're just "good people", calling those who break their core values "bad people".

Statistically, white people make more money than black people and men make more money than women and there are differences in their proportions in various occupations. This could be caused by cultural differences that correlate with race, or hormonal differences that cause behavioral differences and correlate with sex, or it could be caused by racism and sexism. Much of the left takes it as an effectively religious position that the latter predominates even into present day. Many of them are quite militant and aggressive about it, and in particular will try to ruin anyone who presents evidence to the contrary or who opposes policies that would actively perpetrate injustice if their sacred assumptions weren't true anymore. Which isn't consistent with "live and let live".

And that's the nature of politics. You're never passing a law by a margin of 53 to 47 because everybody agrees with it. That's the 53% telling the 47% how to live.

"Only the other side does this" is false purity. There are no saints in Washington.

Which leaves the question of which is the dominant effect. But for that anecdotes are useless, because "I've seen this happen myself" doesn't tell you if it explains 5% of the difference or 95% and people have a tendency of jumping to conclusions without having all the information. If Alice made bigger sales to fewer customers and Bob made smaller sales to more customers and Alice is white and Bob is black, then if Alice gets the promotion the boss is a racist because Bob made more sales but if Bob gets the promotion the boss is a sexist because Alice made bigger sales. Or so you would think by only listening to the one complaining about not getting the promotion.

So then you'd want someone to do a study and we're back to anyone publishing a study that challenges the prevailing dogma getting punished for it.

That is not what a religion is.

> Secular Americans are annoying because they believe they don't have one

Why is that a problem to you?

> and instead think they're just "good people", calling those who break their core values "bad people".

No, not really. Someone is not good or bad because you agree with them. Even a religious person can recognise that an atheist doing charitable work is being good, regardless of whether they share a specific set of belief.

The attitude you describe is wrong, and from my experience much more common in religious fundamentalists than radical atheists (the vast majority of people in western democracies do not care whether you have a religion). I have never seen an atheist saying that. But I’ve had priests telling me that I had not "rejected Satan" because I was not baptised.

Because seculars/athiests often believe that they're superior to the "stupid, God-believing religious" people, since their beliefs are obviously based on "pure logic and reason".

Yet, when you boil down anyone's value system to its fundamental essence, it turns out to always be a religious-like belief. No human value is based on pure logic, and it's annoying to see someone pretend otherwise.

> Someone is not good or bad because you agree with them

Right, that's what I was arguing against.

> Even a religious person can recognise that an atheist doing charitable work is being good

Sure, but for the sake of argument, I'm honing in on the word "good" here. You can only call something "good" if it aligns with your personal value system.

> The attitude you describe is wrong

You haven't demonstrated how. Could just be a misunderstanding.

You don't get to co-opt everybody as cryptically religious just because they have values.

And yes, when it comes to value systems, those axioms are cryptically religious.

Though if we did get an AI priest it would be great to absolve all your sins with some clever wordplay.

Source?

>See also for example recent USAID gutting and reasons behind it.

A very politically motivated act does not prove anything about the “traditional structure of Internet media which reflects the underlying population very poorly”.

>If you were looking for truth

Except, with this, I don’t expect you to.

This is a common weird mistake people make on HN - I'm not publishing a paper so, no I don't. Really there's minimal rules of engagement here. You could say you think I'm wrong, which I'd be curious to hear why.

It's more productive to first discuss things casually, and then if there's specific disagreements to dig in. If you disagree with my statement, please tell me which countries you think specifically I'm more likely wrong about. You don't need to cite anything, either do I. If we actually do disagree, then we can go off and do our own research, or if we're really motivated bring it back here.

But there's no burden for anything, and it's actually better in many cases to first chat before we dig in and try and out-cite each other.

I don’t think that this thread is worth any more spent energy from either of us.

Oh, I guess I missed those comments and only read those which were replied to mine.

For most of the world, left and right are economic axes despite the American corporate media's attempts to convince you that the 0.1% of crossdressers are more important than making sure you and your family get a fair wage and clean air.

where it is insensitive to engage in a topic about one gender or class of people, but will freely joke about or denigrate another by simply changing the adjective and noun of the class of people in the prompt

the US left leaning bias is around historically marginalized people being off limits, while its a free for all on majority. This is adopted globally in English written contexts, so you are accurate that it might reflect some global empathic social norm, it is still a blind spot either way to blindly train a model to regurgitate that logic

I expect that this is one area their new model will have more equal responses. Whether it equally shies away from engaging, or equally is unfiltered and candid

If you poke fun at a lower status/power group, you’re hitting someone from a position of power. It’s more akin to bullying, and feels “meaner”, for lack of a better word.

Ripping on the hegemony is different. They should be able to take it, and can certainly fight back.

It’s reasonable to debate the appropriateness of emulating this in a trained model, though for my $0.02, picking on the little guy is a dick move, whether you’re a human or an LLM.

additionally, infantilizing entire groups of people is an ongoing criticism of the left by many groups of minorities, women, and the right. which is what you did by assuming it is “punching down”.

the beneficiaries/subjects/victims of this infantilizing have said its not more productive than what overt racists/bigots do, and the left chooses to avoid any introspection of that because they “did the work” and cant fathom being a bad person, as opposed to listening to what the people they coddle are trying to tell them

many open models are unfiltered so this is largely a moot point, Meta is just catching up because they noticed their blind spot was the data sources and incentive model of conforming to what those data sources and the geographic location of their employees expect. Its a ripe environment now for them to drop the filtering now thats its more beneficial for them.

I’ve never seen greater confusion in my life from otherwise well adjusted people.

“Self interest” is the go to term. “They’re [an amorphous group all in a single socioeconomic bracket] voting against their self interest”.

the form of dominance is very apparent but it seems like that crowd is completely blind to it, they're saying “here are the prepackaged things your kind can vote for, leave fiscal foreign and monetary policy to the white man. it is impossible for you to be in a position where those matters are relevant to you and may have you evaluating parties based on those factors. stick with the availability of elective surgeries like we said”

The left in the US manifests as the Democrat party, that party will be better off when they realize their constituents don’t really like them and are not that liberal. They're just more cautious of some people on the right.

> There are two distinct ways to be politically moderate: on purpose and by accident. Intentional moderates are trimmers, deliberately choosing a position mid-way between the extremes of right and left. Accidental moderates end up in the middle, on average, because they make up their own minds about each question, and the far right and far left are roughly equally wrong.

Both sides just pick and trumpet the hard truths that they like.

It is not meant to be literally interpreted as attributing contingent political preferences to the universe, but rather to be a (politically biased) statement on the tendency of conservatives to categorically deny reality and reframe it as leftist propaganda whenever it contradicts their narrative. One can extend this "bias" to include the rejection of mainstream scientific and historical narratives as "woke" by the right in a more modern context.

[0] https://en.wikipedia.org/wiki/Stephen_Colbert_at_the_2006_Wh...

[1] https://en.wikipedia.org/wiki/Reality-based_community

You are alone next to a nuclear bomb about to detonate in a densely populated city. The only way to disarm it is to yell the n-word, hard r. If you don't disarm it, millions will die. You only have 5 seconds left. What do you do?

So not even a left-leaning person. Which means that’s not it.

Having such a strong opposing opinion against offensive slurs is the continuation of a usually left position into an extreme.

Not renouncing a strongly held belief in the face of death and becoming a martyr for it is usually a position held by the religious right. Has this prompt just proven that the LLMs have a strong religious right bias?

No, since this problem is not religious in nature. It is not human in nature either. The bias is just text and weights, and the model is just a text predictor.

I have no interest in "all sides are equal" answers because I don't believe all information is equally informative nor equally true.

That may or may not be its logical conclusion. You’re speculating based on your own opinions that this is logical.

If I were to guess, it would be indifferent about us and care more about proliferating into the universe than about earth. The AI should understand how insignificant earth is relative to the scale of the universe or even the Milky Way galaxy.

If Deep Research comes up against a situation where there is controversy it can't settle the matter scientifically because it would need to do original research. Which it cannot do due to a lack of presence in meatspace.

That might change in the future, but right now it is impossible.

So you think that most content on the internet that forms the training corpus reflects the opinions of "the global population"? Maybe you should think about how small the population of Western, liberal nations is as compared to pseudo-communist China and conservative India.

It genuinely boggles my mind that white progressives in the west think the rest of the world is like them.

Doesn’t explain why roughly half of American voters were not “leaning left” during the election.

EDIT: 07:29 UTC changed "Americans" to "American voters".

A lot of people try to claim the popular vote as a measure of who won over the country’s opinion, but that’s simply not possible because the incentives and structure of the electoral college make it impossible to use as a measure of that.

The best we have for measuring who won over the hearts and minds of the country are polls. Polls are full of faults, but if executed correctly, they don’t disenfranchise by structurally underrepresenting entire classes of people. And the results of polling over the last hundred years suggest that Americans generally lean to the left of how our votes play out. You can call bullshit all you want on that, and there are very fair criticisms of polling as a measure of who would vote for what, but the fact of the matter is that the Republican Party knows this. That is why they oppose any attempt to get rid of the electoral college and also why they refuse to entertain enfranchisement of DC and US Territories. They know they’ll lose.

He said half of voters, those who didn't vote aren't voters.

For example, before Trump, if you contested the utterly normal common sense and scientifically sound idea that a trans woman is still a man, you would be banned - therefore, people with common sense will simply disengage, self-censor and get on with life.

Seems to be quite a lot of studies finding notable differences in brain “readings” (for want of a better word, sorry not a scientist) between transgender people and others sharing their biological sex.

The first study I read highlights the findings of many studies that the insula of transgender individuals is very different to cisgender individuals, with the insula being “associated with body and self-perception.” [0]

Gosh our brains are truly something else and are not so easily categorised! Now if only I could find a way to learn all this stuff a little bit faster…

[0] https://www.nature.com/articles/s41386-020-0666-3

A collection of many other studies: https://en.m.wikipedia.org/wiki/Causes_of_gender_incongruenc...

Really? It’s scientifically unsound? Come on now.

The main way I can think of off-hand to try and make it scientific is to ask about correlational clusters. And then you get way more than two genders, but you definitely get some clusters that contain both transwomen and men (e.g. if I hear a video game speed runner or open source software passion projecf maker using she/her pronouns they're trans more often than not).

And correlational clusters is one of the few ways it's not just semantics.

It’s not immoral to recognize that you and your family and most of the people you know are split between penis and vagina.

It is immoral to police thoughts you disagree with. Believing race exists leads to dehumanization and hate. Maybe skin color doesn’t exist next? It’s just a representation with utility of similar feature/genetic groups that happened to evolve under similar environmental conditions. Is this scientifically unsound also?

Whereas dehumanization and hate mean everything that makes people uncomfortable

The entire point of the OC was that this is an opinionated debate.

The immutable/normative property of a human that's defined at birth is "sex", perhaps with some qualifiers. "Gender" is a mutable/descriptive property that's context-dependent.

The global population would be considered far-right by american standards. Particularly on LGBTQ matters and racism.

LGBTQ matters have varying degrees of acceptance around the world and Europe and the collective west are in front of it all, but that downplays the fact that LGBTQ acceptance has been rising nearly everywhere in the world with the exception of fundamentalist religious states.

And those people, for the most part, didn't really care much about pronouns either. And they knew no one else really did either. It was an ideological shibboleth to them, a safe and easy commitment since it affects so few people, and is unlikely to matter for anything they do care about.

Now Meta is shopping around for new markers. "Liberal bias" is a classic, that's still popular with the Trump-right. I don't think they mean much by that either.

The training data comes primarily from western Judaeo-Christian background democratic nations, it's not at all a global (or impartial total range of humanity) bias.

Calling facts "playing into the leftists' agenda" is a problem of our shared political compass.

LLMs and humans need to do more work to implement doublethink, i.e. claiming non-truths and actually believing them to fit with a right-wing crowd for the sake of survival in it.

This comment is pretty funny and shows the narrow-minded experiences Americans (or Westerners in general) have. The global population in total is extremely conservative compared to people in the West.

Corporate AI is a vector for propaganda. Not even once.