How would they prove this? Is it safe to say that each article used has a nearly meaningless influence on the weights?
Could this be used as a defense? Perhaps train a (smaller) model, remove a single article, and show how it doesn't influence performance?
It’s not like it’s ok to violate copyright if you don’t compete. It’s still illegal to take a NYT article and print it on a t-shirt.
The issue is that copyright law doesn’t prevent the kind of model training as there’s no clearly derived work. I don’t think that’s been tested in courts yet, but I expect it won’t be found to be copyright because there’s other precedent that influenced is not infringement.
This issue is ultimately going to come down to the transformative clause of fair use. The fact is that the _model_ is unquestionably a transformative product of the inputs, and a judge ruling otherwise is going to cause a cascading shitstorm of litigation and put a chill through the creative economy. The outputs of the model under certain conditions can be guided towards copyright infringement, and any sane ruling will focus on protecting rightsholders from overly derivative model outputs. In all likelihood the precedent will be that the standard for being transformative will be raised for "algorithmically generated" content, and the people who distribute that content will still be fully liable in the event of infringement, with "I didn't know, the AI did it" not being an acceptable defense.
If I read five calculus textbooks and write a new one, I don't think that's derivative content (or maybe it is?) Seems like that's what an LLM does - read many works, write a new work.