Of course it's not a good thing. General-purpose AI should not be overridden or forcefully trained to favor one political view over another.

There's no way to avoid the bias, whether it's because you chose a different training set, reinforced different pathways, or put blocks in place on certain topics.

I'd rather the authors be explicit in where they are putting their fingers on the scales rather than just relying on "Guess we got lucky".

> There's no way to avoid the bias,

How about collecting a representative sample of all data for your training data?

Or at least, trying to do that as best you can.

Saying "there's no way to avoid the bias" is just an excuse to get away with being biased, in my view.

Let's say that white nationalism is a tiny fraction of ideas online. Significantly less than 0.1%. Now, you randomly sample the internet and do not collect this idea into your training set. Do you adjust your approach to make sure it's represented (because as reprehensible as it is, it is the reality of online discourse in some places?)

I genuinely believe that it's all going to be biased -- there are no unbiased news or media outlets -- and the sooner you recognize everything is biased, the sooner you can move on to building the tools to recognize and understand that bias.

Asking "why can't we strive to build an unbiased outlet" is to me like asking "why can't we build a ladder to the moon". It's an interesting question, but ultimately should lead you to "Well, why do you want that, and your approach is impossible but the outcome you want might not be."

Perhaps you can take a representative (i.e. random and statistically significant enough) sample of the population and ask them their opinion about certain (especially controversial) pieces of your training data, then adjust your training data to weigh more heavily or less heavily based on these evaluations.

That's just one idea that occurred to me from the top of my head, but I'm sure there are research scientists who can devise a better method than what I just came up with in 30 seconds.

> Let's say that white nationalism is a tiny fraction of ideas online. Significantly less than 0.1%. Now, you randomly sample the internet and do not collect this idea into your training set. Do you adjust your approach to make sure it's represented (because as reprehensible as it is, it is the reality of online discourse in some places?)

Sure. Otherwise you're in for a dangerous (and perhaps immoral) slippery slope. But it should be represented only as much as it is significant. Obviously you should not train your AI to weigh these ideas as much as others that are more prevalent. If it's only a tiny minority of the population that have such opinions, that should be reflected in the data (so that there is proportionally less data to account for these ideas).

One would think that a sufficiently intelligent AI would not end up being a white nationalist, though (I'm not talking about current LLM technology, but perhaps some future version of it that is capable of something akin to self-reflection or deep thought).

> I genuinely believe that it's all going to be biased -- there are no unbiased news or media outlets -- and the sooner you recognize everything is biased, the sooner you can move on to building the tools to recognize and understand that bias.

News and media outlets are biased, yes, of course. The content from these sources is not generated from the population in general.

That doesn't mean it's impossible to generate an unbiased sample of data (at least, up to a certain margin of error, depending on effort expended).

And it gets worse. For instance, trans men have a totally different experience in rural vs coastal America vs Europe vs Africa. To get an AI who can speak confidently on what it is like to be trans male in those places will require even more interviews.

An that's before we get into set intersection territory. Take a simple example of being gay or straight, Black or white. Each of them is separately a unique experience. But being gay and white in America is very different from being gay and Black in America -- the two identities create 4 different intersections.

Now, you could say, "My AI simply will not speak about the experience of gay Black men, and the challenges/perspectives from that community", but then you've introduced a bias.

You could say, "Well, we'll go out and interview people from every set then, make sure we're covering everyone!" But where then do you stop sampling? Each additional modifier adds exponential complexity -- gay Black men from New Orleans will have a different experience from gay Black men from Lagos.

No, my approach is asking all types of people about the experience of minority folks, including those minority folks (we are all minorities in some aspect, even if this aspect is uninteresting).

> for instance, if you ask a statistically significant sample of the population about what it is like to be a trans man, you are going to (...) be asking a bunch of people who have no idea what it is like.

Those people can answer that they don't know what it's like to be trans.

If somebody comes up to me and asks me: "what is it like to be trans?". My answer would obviously be: "how the hell should I know? I'm not trans".

But trans people can answer what it's like to be trans.

> And it gets worse. For instance, trans men have a totally different experience in rural vs coastal America vs Europe vs Africa. To get an AI who can speak confidently on what it is like to be trans male in those places will require even more interviews.

Yes, you can only spend a limited amount of effort towards the goal of being unbiased. The goal is to be unbiased as possible given that limited amount of effort.

It's still better to make X amount of effort to be unbiased than zero effort.

This is also something that can be improved over time, as better ideas and methods become available regarding how to measure and decrease bias.

Perhaps even an AI can be used to detect these biases and reduce them as best possible.

> Now, you could say, "My AI simply will not speak about the experience of gay Black men, and the challenges/perspectives from that community", but then you've introduced a bias.

Or perhaps the AI can simply answer based on the information it was trained on, making a best guess as to what that would be like, taking into account all the data that was available to it and how that data was weighed to be as unbiased as possible.

> You could say, "Well, we'll go out and interview people from every set then, make sure we're covering everyone!"

No, I think you are making a significant mistake in this reasoning. There is no "every set". There is only one set. And that is the set of all people.

> But where then do you stop sampling? Each additional modifier adds exponential complexity -- gay Black men from New Orleans will have a different experience from gay Black men from Lagos.

What modifier? There is no modifier. "SELECT RANDOM(x%) FROM TABLE all_people" (or whatever the imaginary SQL syntax would be) :)

So you are therefore biased. You have a finite set of resources, and you are choosing to allocate them in a particular way. That is bias.

You could equally choose to allocate those resources away from the majority, which would also be bias. Any time a human is making an editorial decision about how to allocate resources, you are introducing bias.

> So you are therefore biased

Yes, but significantly less than before. Which is the goal.

> You have a finite set of resources, and you are choosing to allocate them in a particular way. That is bias.

That "particular way" is to give more weight to opinions that are under-represented in your training data and give less weight to opinions that are over-represented in the training data.

This is called "removing bias".

> You could equally choose to allocate those resources away from the majority, which would also be bias. Any time a human is making an editorial decision about how to allocate resources, you are introducing bias.

So, in your view, bias can only increase, it can never decrease?

Even if that were so, you are admitting that not all data is equally biased. Which means that it is possible to feed less biased data to an AI.

And the goal is not for "a human" to make an editorial decision. It's for the opinions used for the training data to be representative of all people, weighed according to (a representative sample of) these people (so you wouldn't be giving more weigh to the opinions of one person versus another).

> Saying "there's no way to avoid the bias" is just an excuse to get away with being biased

And specifically to the implied idea that there is an unbiased target that is achievable.

I don't think we can achieve 0% bias, I agree with you on that. But I think that, if you decide to spend some amount of effort (let's call this amount "X"), it is possible to reduce bias compared to if you spent zero amount of effort.

And that furthermore, if you spend a "Y" amount of effort where X<Y, then you can reduce even more bias.

Obviously, at a certain point this would have diminishing returns, so presumably there is some sweet point where, even though you can't be 100% unbiased, you can at least say you made a reasonable effort to be unbiased and that your remaining sources of bias are unintentional (and probably, almost just as likely to go in one direction vs another).

To bring the conversation back to the original topic, I think ChatGPT/InstructGPT is actually being actively biased towards one political side as a side effect of RLHF being done by people from OpenAI, even if this bias is being introduced unintentionally.

It would be much better, for example, if OpenAI could somehow accomplish RLHF using a sample of its users as the AI trainers.

It would still be a far cry from reaching 0% bias asymptotically, but it would already be an improvement, I think, as I think its users are a more representative sample of the population than OpenAI employees.

It's all about context.

How about instead of preferring the LLM with "woke" opinions, you would prefer an LLM that was simply trained to avoid controversial topics?

That way, you could use it for your product while still avoiding both bias and media/government backlash.

Are you aware that by being biased towards "woke" opinions you are basically alienating about 50% of the population or so?

I agree that it alienates people, but the choice is less between alienating half and alienating no one but more alienating half and alienating another half that includes the media and the law. I'd use the same strategy if I worked in China: business is business and money trumps theoretical concerns about free speech and open dialogue.

So you're saying that if your LLM is unbiased then you are alienating the other half that includes the media and the law?

That's actually very telling.

Though, in fairness, an unbiased model would probably end up alienating closer to 100% of people instead of any particular half of them.

Why?

It's pretty clear customers reject woke content.

You: Perfectly acceptable.

Yes, this is rhetoric, but it's very valid rhetoric. Extreme view tend to get far more print time then their actual occurrence IRL. Your learning model only cares about how much data is put in, so when you get a few billion pages written up by these extreme topics they can bias the model.

But the fact is in a representative democracy favoring viewpoints that destroy democracy is suicide.

> You: Perfectly acceptable.

Why would the general AI say that?

All examples I've seen of that kind of speech from LLMs were due to them being specifically prompted to generate such a response. It's not like the AI decided to say that on its own, in a completely unrelated conversation.

In fact, it wouldn't make sense if the AI did that on its own, would it? Because the AI reflects the data it was trained on and we know that almost nobody is a Nazi.

> Extreme view tend to get far more print time then their actual occurrence IRL.

Yes, I understand that. We live in a crap society. But I'd argue we should strive to educate people on why an LLM can answer like that, not censor it arbitrarily.

There is an infinite amount of stupid or bad things an LLM can answer, depending on the prompt you use, so I would argue that we should just learn to accept that "stupid prompt = stupid answer" rather than trying to make the LLM not answer anything that might be the slightest bit controversial.

> But the fact is in a representative democracy favoring viewpoints that destroy democracy is suicide.

But I'm not arguing for favoring those viewpoints, am I? I am arguing for AI to be unbiased.

I hope you begin to see the problem at hand.

Yes, almost all data is biased... of course.

Some data is 100% biased. Some data is 1% biased.

How about we try to collect data and then weigh it such that what we feed to the AI during training is as unbiased as possible, given a certain amount of effort?

You know that you can actually influence what data you feed to the AI, right? Or how much the training takes some data into account vs some other data, I guess.

You know that you can create a metric for measuring bias, right?

You know that even if you are not capable of being 100% unbiased, you can work towards that goal, right?

You know that there are plenty of smart people who can come up with ideas for eliminating (or mitigating) sources of errors when measuring bias, right?

I hope you begin to see the solution at hand.

Yes, and no.

So, lets go back in the past and do data collection in 1840 from citizens with the right to vote. We'll take one sample from New York City and the other from Mobile Alabama. Now what do you think happens when you query that dataset on views about slavery? Your data is inherently biased. In fact one could say there is no middle ground here.

My view is that a measure of "bias" should reflect what a representative sample of the entire population [1] would answer if you asked them how biased the AI is.

Of course, if you live in a historical context where slavery is socially acceptable, then the answers the AI gives you will reflect that environment. It's no different from raising a human person in that same environment.

The problem is, you can't necessarily know whether something is good or bad without the benefit of hindsight.

Thinking you know better than everyone else and then imposing your view may just serve to magnify your mistakes.

However, one would think that, once we have that technology, a sufficiently intelligent AI would start to have opinions of their own about what is moral/ethical vs what isn't, that isn't strictly a representation of the training data.

[1] of the world even, if that's the target market for the AI.

...such as whether trans people are deserving of equal rights, or whether or not the 2020 election results were fraudulent (which, if one reads TFA, were the cited complaints)