How would they prove this? Is it safe to say that each article used has a nearly meaningless influence on the weights?
Could this be used as a defense? Perhaps train a (smaller) model, remove a single article, and show how it doesn't influence performance?
It’s not like it’s ok to violate copyright if you don’t compete. It’s still illegal to take a NYT article and print it on a t-shirt.
The issue is that copyright law doesn’t prevent the kind of model training as there’s no clearly derived work. I don’t think that’s been tested in courts yet, but I expect it won’t be found to be copyright because there’s other precedent that influenced is not infringement.
This issue is ultimately going to come down to the transformative clause of fair use. The fact is that the _model_ is unquestionably a transformative product of the inputs, and a judge ruling otherwise is going to cause a cascading shitstorm of litigation and put a chill through the creative economy. The outputs of the model under certain conditions can be guided towards copyright infringement, and any sane ruling will focus on protecting rightsholders from overly derivative model outputs. In all likelihood the precedent will be that the standard for being transformative will be raised for "algorithmically generated" content, and the people who distribute that content will still be fully liable in the event of infringement, with "I didn't know, the AI did it" not being an acceptable defense.
If I read five calculus textbooks and write a new one, I don't think that's derivative content (or maybe it is?) Seems like that's what an LLM does - read many works, write a new work.
Fan art and fan fiction are derivative without copying sequences of words
Computers and networks have been around a long time. These issues have been given a good workout.
“Fair Use” can apply to essentially any of the exclusive rights under copyright, including making (with or without distributing) a copy or derivative work.
> Probability models really just aren’t copyrightable to begin with.
“Probability models” that are built from copyrightable works either:
(1) have a sufficient human creative input to be copyrightable on its own (in which caee it still may infringe copyrights applicable to its source material as a derivative work), or
(2) do not have a sufficient human creative input to be copyrightable, and thus are a form of mechanical copy of their data from which they are developed (which, to the extent it either is, in aggregate, a copyrighted work, and/or contains copies of other copyrighted works, is protected by one or more copyrights, which do not cease to apply to the mechanical copy, and which an unlicensed mechanical copy would violate unless it fell into an exception like Fair Use.)
You're halfway there with #2. The output is not copyrightable, but unless you can actually point to a sequence of words from the original it can't be infringing.
They absolutely are copying in the course of training the model, and they are doing something that often looks a lot like copying when producing output with the model.
> Style (which can be described as a probability model) is not copyrightable.
Style is not all that can be described in an LLM's "probability model", otherwise LLM models would never be able to reproduce content from the training set, which they do.
> The output is not copyrightable, but unless you can actually point to a sequence of words from the original it can’t be infringing.
While verbatim copying (“a sequence of words from the original”) is infringing, mechanical copying with a mechanical substitution filter (even a wickedly complex one) is also copying, and violates copyright.
And ChatGPT is quite capable of producing verbatim text from its training set.
Only in the sense that a Cisco router is copying in the course of sending me the article, which we've all agreed doesn't count as infringement.
The bigger problem is that the plaintiff has to show it is more likely than not that that sequence of words came from their text and not some other source, which is going to be obscenely difficult.
> And ChatGPT is quite capable of producing verbatim text from its training set.
Granted, and then the copyright holder could sue for infringement at that point. Exactly whom he should sue is a more difficult question.
Look, I get it: you want copyright to allow authors to say "you can't do that with my work"; I'm sympathetic, but it just doesn't give authors that power.
But it does count as infringement, if its not explicltly or implicitly licensed (because it is necessary to a use that is licensed or necessary to a use that does not itself require a license but is the normal use for for which a licensed copy, which you have, is sold and used) and it is not itself Fair Use (usually, because it is necessary in the course of a use which is itself Fair Use.)
> The bigger problem is that the plaintiff has to show it is more likely than not that that sequence of words came from their text and not some other source, which is going to be obscenely difficult.
The plaintiff has to (1) show facts from which a judge concludes that a reasonable jury might conclude that, and (2) get the jury to conclude that.
This can be difficult, and it might not be trivial in this case, but in practice the combination of opportunity and non-trivial similarity tends to put more weight on the defense to show a convincing alternative explanation. Outside of conclusions that a judge views as conpletely unreasonable based on the facts, the civil burden of proof boils down to what a jury feels is more likely, not some.
> Granted, and then the copyright holder could sue for infringement at that point. Exactly whom he should sue is a more difficult question.
Well, who else might be liable depends on the specific circumstances, but with OpenAI controlling the whole course between the first copy of the copyrighted work and the infringing end product, and doing it all for commercial gain, that OpenAI wpuld be on the hook, civilly and potentially criminally for any infringement, is clear.
> Look, I get it: you want copyright to allow authors to say "you can't do that with my work"; I'm sympathetic, but it just doesn't give authors that power.
I want copyright to be both more limited in its exclusive rights and either shorter or costlier to the copyright holder than it is. But what I want is not what the law is.
I don't think you've thought about this hard enough
The transient copy isn’t for a licensed use, and whether it is a Fair Use is specifically the subject of debate. So this really is basically an admission that but for the potential applicability of Fair Use, the use of the copyrighted mmaterial in training is a violation of copyright.
Also, even if the training is fair use, that doesn’t mean that the copies of the source material produced by OpenAI and distributed to their customers using the model are fair use. Just becaue making the tool is a transformative fair use doesn’t mean using the tool to generate copies of the material which was used to train it, which are significantly less trandormative than the model itself, are Fair Use. (And the fact that one of the functions that the model is used for is this commercial, for profit by the maker of the model, copying of the source material is – as much as I believe AI model training on its own is quite likely to generally be fair use – an argument against the model training being fair use in this csase.)
> Computers and networks have been around a long time.
True, and commercially producing and delivery copies of copyrighted works in a manner which substitutes for the original work in the marketplace, no matter what intermediate steps go into doing that, and no matter that computers or networks are used in those intermediate steps, is pretty much the clearest case of violation of copyright you can get.
> These issues have been given a good workout.
Some of them have, some of them have not. Whether and in what conditions training an AI on source material that may be subject in aggregate to a compilation copyright by someone else, and which consists further of individual works that have their own copyrights, might be “fair use” is not one of the issues that have been given a good workout. Neither – because producing predictive models in that way has not previously been common – has whether, unlike other intermediate tool use, using such a model in the course of doing what would otherwise be an infringement by producing a copy of specific copyright-protected works, commercially, for a customer at their request, is no longer a violation because the use of the model somehow isolates it from liability,
I say if people want to come up with bullshit lawsuits, we should play into them and force those people to suffer the consequences of said bullshit.