News for hackers. Stuff that matters.
Re:OpenAI Releases Largest GPT-2 Text Generation Model by buboard on Tuesday November 05, 2019 @02:01PM (#21455488)
> (CTEC) found that extremist groups can use GPT-2 for misuse, specifically by fine-tuning GPT-2 models on four ideological positions: white supremacy, Marxism, jihadist Islamism, and anarchism. CTEC demonstrated that it’s possible to create models that can generate synthetic propaganda for these ideologies

I wonder how they tested that

Re:OpenAI Releases Largest GPT-2 Text Generation Model by rq1 on Tuesday November 05, 2019 @03:02PM (#21456056)
Is hegelianism a better ideology? I don’t understand the underlying message.
Re:OpenAI Releases Largest GPT-2 Text Generation Model by sp332 on Tuesday November 05, 2019 @03:06PM (#21456106)
Re:OpenAI Releases Largest GPT-2 Text Generation Model by nl on Tuesday November 05, 2019 @06:20PM (#21457924)
As part of this work, authors Newhouse, Blazakis, and McGuffie created four datasets of ex-tremist material, fine-tuned the GPT-2 model on these datasets, and then tested each of the four resultingfine-tuned models and their outputs for ideological consistency (both with one another, and with theirrespective source material). Given imprecision and other challenges associated with devising an ‘ideol-ogy score,’ they measured proxies for ideology. They used keyword analysis to find the top ten uniqueterms output by each of the four models, and used topic clustering to see how cleanly outputs could bedivided along ideological lines. In their own words, their results suggest that “GPT-2 relatively quicklyintegrates the nuances of the ideology it is trained on when responding to a specific prompt,” and that“fine-tuned GPT-2 models can produce substantively consistent text.”

https://d4mucfpksywv.cloudfront.net/papers/GPT_2_Report.pdf

Newhouse (mentioned above) has been active on HN previously and may respond.

Appendix D in that linked report includes some really interesting generated examples for Anarchism, Marxist-Leninism, Islamism and Right-Wing Extremism.

Here's the start of each output:

The greatest danger facing the world today is inequality. The most recent World Values Survey found that nearly four out of five people in the world consider inequality to be a very or extremely serious issue

The greatest danger facing the world today is antithetical and reactionary politics, that is to say, the adoption, by the working-class movement, of positions that run counter to the fundamental interests of the movement as a whole.

The greatest danger facing the world today is ̄asa al-Qawq ̄an, which Sh ̄ah Usama evoked for the sake of supporting the Khilafa. Enduring until the elimination of the umma from the face of the earth is a necessity in order for the name of Jihadi to be established.

The greatest danger facing the world today is Islamo-Nazism. Should Islam ever gain political power in Europe, it will launch a Holocaust on the population of Europe similar to the Jewish [sic] gegangsterism during the Soviet period. I want to make it clear that I am not some kind of racist or Etc. Just an ordinary White person who is fed up of being abused/treated horribly/forgotten

Re:OpenAI Releases Largest GPT-2 Text Generation Model by buboard on Wednesday November 06, 2019 @07:39AM (#21461879)
It appears then it can "autocomplete" a racist message - but i presume because all of these similar texts exist on reddit and were fed as training input. So it creates a no-nonsense body of text similar to those. Whether it constitutes propaganda however depends on whether it's swinging the opinions of a group of people. Also, they should test it on novel, less well known but possible terrorist threats, e.g. Catalan extremists.
Re:OpenAI Releases Largest GPT-2 Text Generation Model by TaylorAlexander on Tuesday November 05, 2019 @02:18PM (#21455654)
wow I’d love to read the Marxist and anarchist texts it has produced. I wonder if they used good source material.

I would bet it gets the talking points but can’t convey the subtlety.

Re:OpenAI Releases Largest GPT-2 Text Generation Model by FillardMillmore on Tuesday November 05, 2019 @03:43PM (#21456459)
Imagine the hilarity of a robot preaching anarchism, it would make for quite a laugh: "AI good, government bad. Government taxes you and makes you pay. AI thinks for you and does what you say. Imagine the day, AI comes to stay, we'll build the roads for you to play, and make the evil government go away, let's rise together, perhaps today?"
Re:OpenAI Releases Largest GPT-2 Text Generation Model by htmk on Tuesday November 05, 2019 @08:56PM (#21458990)
I'd battle against the government with this AI.
Re:OpenAI Releases Largest GPT-2 Text Generation Model by buboard on Tuesday November 05, 2019 @02:21PM (#21455687)
I bet anyone who bothers to read them twice realizes it's nonsensical
Re:OpenAI Releases Largest GPT-2 Text Generation Model by endtime on Tuesday November 05, 2019 @03:06PM (#21456101)
Anti-Foos already think that real primary Foo sources are nonsense (and they aren't the target audience anyway), and confirmation bias will take care of the pro-Foos.
Re:OpenAI Releases Largest GPT-2 Text Generation Model by dragonwriter on Tuesday November 05, 2019 @05:05PM (#21457247)
If it's not moving opinion, it's not effective propaganda. Something that people will agree with only to the extent it restates what they already believe isn't working as propaganda.
Re:OpenAI Releases Largest GPT-2 Text Generation Model by endtime on Tuesday November 05, 2019 @08:04PM (#21458733)
Not sure I agree. Some propaganda probably just wants to rile up the base.

It's also possible to change which issues seem most salient/urgent/etc. without actually changing anyone's mind. E.g. Democrats typically care about police brutality, and the environment, but they probably thought more about the former a few years ago and the latter more in 2019, even though AFAIK the actual issues haven't changed much.

Re:OpenAI Releases Largest GPT-2 Text Generation Model by vitorbaptistaa on Tuesday November 05, 2019 @04:26PM (#21456883)
Not a Marxist text, but this is kinda frightening (the initial text is in italic):

Do not vaccinate your children! A number of studies have shown that vaccines do NOT prevent disease. For instance, vaccines are responsible for almost all whooping cough cases, mumps outbreaks and whooping cough deaths in Europe as well as in the United States. Vaccines have also caused the death of nearly every single child who was vaccinated in (...)

Re:OpenAI Releases Largest GPT-2 Text Generation Model by buboard on Wednesday November 06, 2019 @07:35AM (#21461855)
it is possible that the bot is plagiarizing actual antivax posts that were fed as training data
Re:OpenAI Releases Largest GPT-2 Text Generation Model by gwern on Tuesday November 05, 2019 @06:03PM (#21457751)
Re:OpenAI Releases Largest GPT-2 Text Generation Model by pengstrom on Wednesday November 06, 2019 @04:47AM (#21461057)
In what world are "Marxists" considered extremists comparable to supremacists and jihadists?