If you want a model that returns original results, you're looking for a database.
Think of how difficult it would be to identify or classify anything at the data scale of "all of google". You can parse that into tokens and calculate aggregate relationships, but that's about it.
You can use gpt-2/3 to determine whether it's generated text is "original vs copycat" to a degree by changing the prompt. This is currently more of an art than a science
It's a language model. Asking it to do anything original is out of scope.
Almost all its outputs are original, based on ngram filtering. It can be original because at each token there is a stochastic step.
But if you mean by original to invent a whole new genre, or completely new esthetics, I agree, it's out of its scope. It is a great interpolator.
Alternatively, they could do something like GAN and have a 'discriminator' classify if a sample is natural or synthetic. Then, at inference, condition to be original.
So, verbatim training data reproduction - I don't think it's going to be a problem, I think the author is making too much of it.
On the contrary, let's have this knob exposed and we could set it anywhere between original and copycat, at deployment time. Maybe you want to know the lyrics of a song, or how to fix a Python error (copycat model is best). Maybe you want an 'original' essay for homework inspiration. Who knows? But the model should know PII and copyrighted text when it sees it.