Can it do so more than a human can?
I think that's the key here. If an AI is no more precise than a human telling you about the news article they read today then ChatGPT learning process probably can't be morally called copying.
Feeding someone else data into your system is usually a violation of copyright. Even if you have a very "smart" system, trying to transform and obfuscate the original data.
In some circumstances, yes, but often it's not, especially if you're not continuing to store and use it (which OpenAI isn't).
So I'm a living breathing copyright violator. As a person I should be banned.
Fortunately, copyright is a bullshit fictitious right with no basis in natural law. So I don't lose much sleep over it.
The court could ask to show the training dataset.
And if OpenAI were selling the reproductions, that would be infringement. But that's not what's happening here. It's selling access to a system that can do countless things.
When you tell people about some news article you read earlier you repeat it exactly verbatim? You also give this out to potentially millions or hundreds of millions of people for commercial purposes?
Furthermore, there's Google research on extracting training set data from models. More specifically, Google found out that if you ask GPT to repeat the same word over and over again, forever, it eventually starts printing fully memorized training set data[0]. So it is memorizing stuff, even if it's not regurgitating it.
[0] When told of this, OpenAI's response was to block conversations with large amounts of repeated words in them.
I'd say the only real reason the Piratebay links thing you mentioned is not the norm is purely because those media sources have done a better job of striking fear into people doing that, so it's gone more underground. I.e. they're better terrorists.
There's no fundamental, moral reason why Piratebay links being posted and raised to the top would be wrong.