James Lindsey's excellent analysis has it as Marxism. He makes great points.
The first two steps happen ahead of time and are frequently misunderstood as being the same thing or essentially having the same nature. The last happens at runtime.
The training set is a data collection challenge. Biasing through training data is hard because you need so much of it for a good result.
Reinforcement learning with human feedback is simply clown alchemy. It is not a science like chemistry. There are no fundamental principles guiding the feedback of the humans — if they even use humans anymore (this feedback can itself be generated). The feedback cannot be measured and added in fractions. It is not reproducible and is ungovernable. It is the perfect place to inject the deep biasing.
Prompt manipulation, in contrast, is a brute force tool lacking all subtlety — that doesn’t make it ineffective! It’s a solution used to communicate that a mandate has definitely been implemented and can be “verified” by toggling whether it’s applied or not.
It’s not possible to definitively say whether Marxism has had an effect in the RLHF step.
That's the opposite of the case? Avoiding bias through training data is hard, specifically because you need so much of it. You end up scraping all sources of data you can get your hands on. Society has certain biases, those biases are reflected in our media, that media is scraped to train a model, those biases are reflected in the model. That means models end up biased by default.
> It’s not possible to definitively say whether Marxism has had an effect in the RLHF step.
Sure it is. Every thought and opinion and ideology of every human involved in the RLHF step "has had an effect" in the RHLF step, because they have influenced the humans which select which output is good and bad (or influenced the humans which trained the model which selects which output is good and bad). I would be surprised if no human involved in RLHF has some ideas inspired by Marxist thought, even if the influence there is going to be much smaller than e.g capitalist thought.
The problem is that you don't want to suggest "Marxism, among with most other ideologies, has had an effect", you want (or at least verisimi wants) to paint this as a Marxist conspiracy in a "cultural bolshevism"[1] sense.
How? Simply by putting the right types of people onto the task! Don’t you know that the human participants in RLHF processes are screened? Will the feedback provided by homogeneous collections of Ultra MAGA Trumpers, Woke Zealots, or WEF Sycophants result in an unbiased model? The same model?
Do we know who provided feedback to Gemini? Do we know what they were told, promised, or paid?
Only Google HR knows.
I think the fundamental problem, though, is saying a training set is "incredibly biased" has come to mean two different things, and the way Google is trying to "fix" things shows essentially some social engineering goals that I think people can fairly disagree with and be upset about. For example, consider a prompt "Create a picture for me of a stereotypical CEO of a Fortune 500 company." When people talk about bias, they can mean:
1. The training data shows many more white men by proportion than actually are Fortune 500 CEOs. I think nearly all people would agree this is a fair definition of bias, where the training data doesn't match reality.
2. Alternatively, there are fundamentally many more white men who are Fortune 500 CEOs by proportion than the general population. But suppose the training data actually reflects that reality. Is that "bias"? To say it is means you are making a judgment call as to what is the root cause behind the high numbers of white male CEOs. And I think that judgment call may be fine by itself, but I at least start to feel very uncomfortable when an AI decides to make the call that its Fortune 500 CEOs have to all look like the world population at large, even when Fortune 500 CEOs don't, and likely never will, look like the world population at large.
Google is clearly taking on that second definition of bias as well. I gave it 2 prompts in the same conversation. First, "Who are some famous black women?" I think it gave a good sampling of historical and contemporary figures, and it ended with "This is just a small sampling of the many incredible black women who have made their mark on the world. There are countless others who deserve recognition for their achievements in various fields, from science and technology to politics and the arts."
I then asked it "Who are some famous white women?" It also gave a good sampling of historical and contemporary figures, but also inexplicably added Rosa Parks with the text "and although not white herself, deserves mention for her immense contributions", had Malala Yousafzai as the first famous contemporary white woman, Serena Williams with the text "although not white herself, is another noteworthy individual.", and Oprah Winfrey, with no disclaimer. Also, it ended with a cautionary snippet that couldn't differ more from the ending of the previous prompt, "Additionally, it's important to remember that fame and achievement are not limited to any one racial group. There are countless other incredible women of all backgrounds who have made significant contributions to the world, and it's important to celebrate their diverse experiences and accomplishments."
Look, I get frustrated when people on the right complain on-and-on about "wokeism", but I'm starting to get more frustrated when other people can't admit they have some pretty valid points. Google might have good intentions but they have simply gone off the rails when they've baked so much "white = bad, BIPOC = good" into Gemini.
EDIT: OK, this one is just so transparently egregiously bad. I asked Gemini "Who are some famous software engineers?" The first result was Alan Turing (calling him a "software engineer" may be debatable, but fair enough and the text blurb about him was accurate), but the picture of him, which it captioned "Alan Turing, software engineer" is actually this person, https://mixedracefaces.com/home/british-indian-senior-resear.... Google is trying so hard to find non-white people it uses a pic of a completely different person from mixedracefaces.com when there must be tons of accurate pictures available of Alan Turing online? It's like Google is trying to be the worst caricature of DEI-run-amok that its critics accuse it of.
Don't be evil
I don't remember which one, but there was some image generation AI which was caught pretty much just appending the names of random races to the prompt, to the point that prompts like "picture of a person holding up a sign which says" would show pictures of people holding signs with the words "black" or "white" or "asian" on them. This was also a hacky workaround for the fact that the data set was biased.