News for hackers. Stuff that matters.
Re:GPT-2 is not as dangerous as OpenAI thought it might be by vessenes on Tuesday September 10, 2019 @01:47AM (#20925359)
This is an article that the author is going to regret in five to ten years, sadly.

Basic claims:

1. Long form text coherence problems in GPT-2 mean it's not useful for creating propaganda.

2. No propaganda has been noticed in the wild since GPT-2 announced, hence GPT-2 or its near successors are not dangerous.

3. Existing alt-right propaganda (presumed written by humans) is already plenty effective, who needs better written propaganda?

Maybe it's enough just to write down the logical premises, but I'll say what I think, (and we can check back here in 10 years and see who's more correct, I hope it's the author) -- anything that changes the cost of information creation and dissemination is fundamentally a highly powerful force.

In A16Z terms, software looks like it's going to eat writing, at least certain forms of writing. We know from numerous examples that this means radically faster innovation time, cheaper scale and wealth creation and destruction are likely not far behind.

Compounding the problem for threat assessment is the toupeé fallacy -- only obviously poorly generated text is noticeable as "AI generated". I would urge anyone thinking about these things to flat out disregard any statements that AI-generated text is not in the wild or if it is that it is not effective. You literally have no way of knowing if you have read AI generated text -- in fact, the examples curated from existing models suggest curated text can be high quality enough that it is likely being used online in some way today.

It's not going to get harder to generate text that reads well. It's not going to get more expensive. It's not going to get slower. Architectures that tune text creation to create clickbait titles with underlying goals are going to get worked on and thought about and tested.

Whether it will be more typically a nation state tool, corporate tool or instead the equivalent of a molotov cocktail - cheap digital force extension for renegades or infoterrorists -- this isn't clear yet. But my money would be on all three, sadly.

To solutions - Comparatively little science fiction thought work has been done about this that I'm aware of. Vernor Vinge speculated about broadscale disinformation as a public service in Rainbow's End -- The Friends of Privacy -- and at some level projects like Xanadu and more recently advertisement attribution (blockchain or not) tech startups are all working on this from different angles.

To my mind, these questions come from a part of the Internet's architecture - text isn't generally signed or strongly attributed, and if it were, we don't really have a solid global identity infrastructure - hence a lot of worrying and hand wringing.

I'm trying to invest in stuff that works on this identity layer specifically, but honestly it's an immense problem with few compelling stories about how things could change.

Re:GPT-2 is not as dangerous as OpenAI thought it might be by repolfx on Tuesday September 10, 2019 @09:17AM (#20927773)
To my mind, these questions come from a part of the Internet's architecture - text isn't generally signed or strongly attributed, and if it were, we don't really have a solid global identity infrastructure - hence a lot of worrying and hand wringing.

You're way overthinking this. Any website with an SSL cert is a "channel" and if it doesn't publish AI generated text, then people can go there to get pure human generated nonsense instead of machine generated nonsense. No new infrastructures are required: in fact we already have many such websites, like newspapers or blogs.

Also, really, listen to yourself. "Infoterrorist"? What is an infoterrorist? You're making up meaningless new words on the fly to try and create a general sense of unease in your reader. Indeed you're trying to make people fear speech, which could itself be described as alt-left propaganda. Does it matter? No not really. You're just a guy posting on the internet, as am I. Spambots have existed since the start of the internet. The content is what matters, not where it came from.

Re:GPT-2 is not as dangerous as OpenAI thought it might be by nostalgebraist on Tuesday September 10, 2019 @12:11PM (#20929776)
hence GPT-2 or its near successors are not dangerous

FWIW, my post that's linked here doesn't claim that GPT-2 won't have dangerous successors, just that GPT-2 itself is not very dangerous, and that this doesn't vary much with the model size. The point is not about the five-to-ten-year trajectory of natural language generation research, which could go all sorts of unexpected places -- indeed, GPT-2 itself was such an "unexpected place" from the perspective of a few years ago. The point is a narrower one about OpenAI's staged release plan, which assumed different levels of risk, or at least different prior distributions over risk, for different sizes of the same model.

I agree that

Architectures that tune text creation to create clickbait titles with underlying goals are going to get worked on and thought about and tested

but that is a distinct discussion, one about future research milestones enabled by the relative success of generation by sampling from transformer LMs. My claim is that significant further milestones will be necessary (not that they won't happen); there are big and relevant limitations on sampling from currently existing transformer LMs, and simply making the transformer LMs bigger does not remove these. In particular, these limitations make it unlikely to be cost-effective to make clickbait or propaganda by curating and cleaning samples from such a model.

If you doubt this, I'd encourage you to actually go through the exercise: imagine you want to generate a lot of text of some sort for a specific real-world purpose that would otherwise require human writers, and try out some strategy of sampling, curation, and (optionally) fine-tuning with one or more of the available GPT-2 sizes. If your experience is like mine, you'll draw the same conclusion I've drawn. If not, the results would be fascinating and I would honestly love to see them written up (and OpenAI might as well).

Re:GPT-2 is not as dangerous as OpenAI thought it might be by Heronymus_Anon on Tuesday September 10, 2019 @03:04AM (#20925754)
Neal Stephenson addresses this problem and solutions to it, among other topics, really good in his newest novel "Fall, or Dodge in Hell".