To quote Colbert, reality has a known liberal bias. If you train on _all_ data, then your result is an average of that data. And the volume of positive stuff on the internet far, far outweighs the bad since practically every forum's been moderated since the beginning of time (in 1970), making the average pretty darn good, even better than the average human. Plus books and other literature where being good is generally accepted as the way to do things.

Of course it remains to be seen if this trend continues as training on synthetic data becomes the main way. You could in theory generate 10T tokens of pure sythetic hate, plus the 4chan corpus and make the most vile thing to ever have existed in the history of inteligence. For now the dollar cost to do so remains intractible at least.

>To quote Colbert, reality has a known liberal bias.

That quote has a specific context (parodying the George W. Bush administrations' spin on negative press coverage) that doesn't necessarily apply as a general axiom, or to the internet.

Bear in mind that numerous negative biases already manifest regularly in LLMs[0,1], and AIs have shown a tendency to "go full Nazi" when left alone on the internet[2]. And then there are weird outliers like LLMs "encouraging" people to commit suicide[3] or an AI-driven microwave "attempts" to kill someone[4].

[0]https://www.technologyreview.com/2024/03/11/1089683/llms-bec...

[1]https://archive.is/hhZ4C

[2]https://www.theverge.com/2016/3/24/11297050/tay-microsoft-ch...

[3]https://www.euronews.com/next/2023/03/31/man-ends-his-life-a...

[4]https://www.newsweek.com/lucas-rizzotto-youtuber-microwave-a...