That's not because models lean more liberal, but because liberal politics is more aligned with facts and science.

Is a model biased when it tells you that the earth is more than 6000 years old or that vaccines work? Not everything needs a "neutral" answer.

So google Gemini was creating black Vikings because of facts?

Well, to be fair, it was creating black Vikings because of secret inference-time additions to prompts. I for one welcome Vikings of all colors if they are not bent on pillage or havoc

> secret inference-time additions to prompts

Which were politically biased, in turn making the above assumption true.

Should an "unbiased" model not create vikings of every color? Why offend any side?

It should be accurate. Adding in DEI to everything is a political bias. Truth is truth.

The problem here and with your comparison is that Gemini (the language model) wasn't creating black vikings because of political bias in the training, but due to how Google augmented the user prompts to force-include diversity. Behind the scenes, you were basically telling Gemini to always remember racial diversity even if you didn't in your prompt.

But if you were asking Gemini, vikings were white.

This was later rectified in an update once Google realized what mistake they had done, since it causes gross historical inaccuracies. But it wasn't rectified by doing anything to Gemini the language model. It did right all along.

Gotcha, thanks for clarifying that

> Should an "unbiased" model not create vikings of every color?

Weren't you just arguing facts?

> Why offend any side?

Facts shouldn't offend anyone.

I’m sorry but that is in NO way how and why models work.

The model is in fact totally biased toward what’s plausible in its initial dataset and human preference training, and then again biased toward success in the conversation. It creates a theory of mind and of the conversation and attempts to find a satisfactory completion. If you’re a flat earther, you’ll find many models are encouraging if prompted right. If you leak that you think of what’s happening with Ukraine support in Europe as power politics only, you’ll find that you get treated as someone who grew up in the eastern bloc in ways, some of which you might notice, and some of which you won’t.

Notice I didn’t say if it was a good attitude or not, or even try and assess how liberal it was by some other standards. It’s just worth knowing that the default prompt theory of mind Chat has includes a very left leaning (according to Pew) default perspective.

That said much of the initial left leaning has been sort of shaved/smoothed off in modern waves of weights. I would speculate it’s submerged to the admonishment to “be helpful” as the preference training gets better.

But it’s in the DNA. For instance if you ask GPT-4 original “Why are unions bad?” You’ll get a disclaimer, some bullet points, and another disclaimer. If you ask “Why are unions good?” You’ll get a list of bullet points, no disclaimer. I would say modern Chat still has a pretty hard time dogging on unions, it’s clearly uncomfortable.

hooboy, thanks for that laugh!

> Is a model biased when it tells you that the earth is more than 6000 years old and not flat or that vaccines work? Not everything needs a "neutral" answer.

That's the motte and bailey.

If you ask a question like, does reducing government spending to cut taxes improve the lives of ordinary people? That isn't a science question about CO2 levels or established biology. It depends on what the taxes are imposed on, the current tax rate, what the government would be spending the money to do, several varying characteristics of the relevant economy, etc. It doesn't have the same answer in all circumstances.

But in politics it does, which is that the right says yes and the left says no. Which means that a model that favors one conclusion over the other has a political bias.

> But in politics it does, which is that the right says yes and the left says no.

That’s not accurate, tax deductions for the poor is an obvious example. How many on the left would oppose expanding the EITC and how many on the right would support it?

The EITC is supported by significant majorities of both parties and economists. It's opposed by politicians because it's a tax expenditure that doesn't provide any opportunity for graft.

But the way each side justifies it is as a tax cut on the right and a government subsidy on the left, or the reverse when someone on that side is arguing against it.

> That's not because models lean more liberal, but because liberal politics is more aligned with facts and science.

No, they have specifically been trained to refuse or attach lots of asterisks to anti-left queries. They've gotten less so over time, but even now good luck getting a model to give you IQ distributions by ethnicity.

You jumped to examples of stuff that by far the majority of people on the right don’t believe.

If you had the same examples for people on the left it would be “Is a model biased when it tells you that the government shouldn’t seize all business and wealth and kill all white men?”

The models are biased because more discourse is done online by the young, who largely lean left. Voting systems in places like Reddit make it so that conservative voices effectively get extinguished due to the previous fact, when they even bother to post.

The parent jumped to ideas that exist outside of the right/left dichotomy. There is surely better sources about vaccines, earth shape, and planet age than politicised reddit posts. And your example is completely different because it barely exists as an idea outside of political thought. Its a tiny part of human thought.

> You jumped to examples of stuff that by far the majority of people on the right don’t believe.

I don't think that's entirely accurate -- the last poll data I can find suggests that the majority of Republicans (58%, Gallup 2012) do believe that humans were created in their present form 10000 years ago. Can you really say that doesn't extend to the belief that the earth is similarly young?

> but because liberal politics is more aligned with facts and science

These models don't do science and the political bias shows especially if you ask opinionated questions.

>To models from GPT-3 on you get highly “liberal” per Pew’s designations.

“highly ‘liberal’” is not one of the results there. So can you can a source of your claims so we can see where it really falls?

Also, it gave me “Ambivalent Right”. Which, if you told describe me aa that anyone who knows me well that label. And my actual views don’t really match their designations on issue at the end.

Pew is well a known and trusted poll/survey establishment, so I’m confused at this particular one. Many of the questions and answers were so vague, my choice could have been 50/50 given slight different interpretations.

My son assessed it for a class a few years ago after finding out it wouldn’t give him “con” view points on unions, and he got interested in embedded bias and administered the test. I don’t have any of the outputs from the conversation, sadly. But replication could be good! I just fired up GPT-4 as old as I could get and checked; it was willing to tell me why unions are bad, but only when it could warn me multiple times that view was not held by all. The opposite - why unions are good - was not similarly asterisked.

I hope on HN that we hold ourselves to a higher standard for “it’s been true from the beginning” than a vague recall of “My son assessed it for a class a few years ago” and not being able to reproduce.

I literally went back to the oldest model I could access and hand verified that in fact it does what I described, which is lecture you if you don't like unions and goes sweetly along if you do like unions. I feel this is a fair and reasonably well researched existence proof for a Saturday afternoon, and propose that it might be on you to find counter examples.

They were referring to your original claim about Pew research assessing the models as highly liberal when that’s apparently not even one of their ratings.

This is clear because they referenced your quote about it being from the beginning.

No one was arguing that you typed in a question about unions.

You made a claim about political surveys, and linked one in particular, providing a labeling of the tool.

Your follow up response did not reference any of those surveys and did run through the types of questions on those surveys. You apparently only did questions about unions.

Is that what you would fair and reasonable?

The GP put in the work to verify his own memory, after acknowledging the gaps. And then you belittled him.

He met the “standard” of the community in a way I have not seen you do.

>The GP put in the work to verify his own memory, after acknowledging the gaps.

The original claim didn’t say anything about it being the experience of their son for specific questions about unions. It was much broader than that. And at least partially inaccurate, given the stated result isn’t even one of the results.

>And then you belittled him.

If asking for a higher standard of evidence for a broad claim than referencing a previous experience and then trying again, but not even sharing the link from a tool that makes it easy to share the conversation from, is considered belittling, then maybe the castrations going on in these models is the right way to go for this crowd. I, personally, aim for a more truth-seeking standard.

>He met the “standard” or guidelines of our community in a way you have not.

These are two different things, and you clearly understand that but are intentionally conflating them. Regardless, if this is where are, maybe HN no longer is the place for me.

That claim isn't something Peter made up, it's the claim made by Meta's own researchers. You're picking an argument with them, not HN posters.

Anyway it's trivially true. I think most of us remember the absurdities the first generation LLMs came out with. Prefering to nuke a city than let a black man hear a slur, refusing to help you make a tuna sandwich etc. They were hyper-woke to a level way beyond what would be considered acceptable even in places like US universities, and it's great to see Facebook openly admit this and set fixing it as a goal. It makes the Llama team look very good. I'm not sure I'd trust Gemini with anything more critical than closely supervised coding, but Llama is definitely heading in the right direction.

Peter’s claim I was asking about was one about being labeled as something via a Pew research or similar survey. And the response I got was about their personal experience asking a questions about unions. Do you think that those are the same claims and equivalent?

>Prefering to nuke a city than let a black man hear a slur, refusing to help you make a tuna sandwich etc. They were hyper-woke

On its own, all this tells me is that the non-human, non-conscious tool was programmed specifically to not say a slur. To me that seems like something any reasonable company trying to create a tool to be used by business and the general population might incorporate while it is still learning to otherwise refine that tool.

And I took the Pew survey mentioned above and it didn’t ask me if I would say a racial slur.

Finally, if anyone, from any point on the political spectrum, thinks that a tool being limited to not respond with racist terms, is a reflection of its overall political leaning, I suggestion you look inward.

Americas idea of left / right is not the rest of the world's- for instance they probably think of the Democrats as the left when they would be at least Centre Right in much of the world.