News for hackers. Stuff that matters.
Re:Writing for Snobs by pmoriarty on Tuesday September 15, 2020 @08:18PM (#24488247)
"if you drastically reduce the size of the corpus won't you also drastically reduce the output domain to the point that it's no longer useful?"

Usefulness is in the eye of the beholder.

A model trained on, say, all the works of Shakespeare might not create something that can pass for Shakespeare's lost work, but might be inspiring, interesting, or have a Shakespearean quality about it in small doses... which could still be very useful for a writer seeking inspiration.

Incidentally, all of Shakespeare's works make up a not insignificant amount of data, though obviously it's a lot less than what GPT-3 was trained on.

Other prolific authors might also provide enough data to train a model.

It's not clear to me at what point the amount of data one uses becomes too little for useful results, and it would surprise me if there was any consensus on this matter, considering the enormous variety in AI techniques, combination of techniques, and parameters that can be used for training and testing, along with all the different uses one might put the models to after they're trained.

Re:Writing for Snobs by caconym_ on Tuesday September 15, 2020 @08:38PM (#24488375)
> Usefulness is in the eye of the beholder.

I think the context here is the question of what "kinds" of writing GPT-3 and similar technologies can and can't do as well as humans, so when I talk about "usefulness" I'm really talking about whether GPT-3 can do its thing of generating passable text in response to widely varied prompts. If it can't do that in some domain, then it's not going to replace writers working in that domain.

That doesn't mean the output won't be interesting, but it will be far less relevant in the ways that make GPT-3 notable in the first place.