This likely varies by jurisdiction, but in the United States, copyright law covers four exclusive rights:
1. The right to make copies
2. The right to distribute copies
3. The right to create derivative works
4. The right to publicly perform the work
Making copies for personal use absolutely violates the first of these exclusive rights.
https://en.m.wikipedia.org/wiki/Sony_Corp._of_America_v._Uni....
My understanding of the reality:
* Violations of those four rights give copyright holders standing to sue over duplication.
* Fair Use is a defense against that violation.
What I was trying to get at in my first post above is that it is a violation of copyright law as the law is written, and because of this, you expose yourself to lawsuits when you make copies for personal use (assuming anyone could ever find out...I'm not, and never have, argued that copyright law is correct or good). The fact that you can defend yourself using a Fair Use defense is also important and a good point, but still requires hiring a lawyer, going to court, and winning. In my book, this is quite a bit different than saying it's not a violation.
IANAL, though I follow this fairly closely, and I'm not aware of any cases that have established that copying for personal use is Fair Use; I'd be very happy to read through a decision that establishes this!
Copyright law “as the law is written” says fair use is not a violation. 17 USC § 107: “Notwithstanding the provisions of sections 106 and 106A, the fair use of a copyrighted work [...] is not an infringement of copyright.”
Its true that many times fair use will be somewhat harder to predict what a court would decide on than other aspects of copyright law, because the statute and precedent in combination are less clear, but that's not a categorical distinction.
Further, these models are not used to reproduce copyrighted works... they would be one of the least efficient ways to do so.
I also don't think there's a particularly strong argument that training on a book or image decreases the market for the original book or image. To take an example, zero folks that want to read Game of Thrones are asking ChatGPT to generate Game of Thrones for them instead of buying the book.
I think what's really happened here is that artists' work has utility outside of the originally intended audience, that utility largely comes from statistically analyzing the writing, and the debate is about whether or not the utility of those statistics requires additional compensation for the artists. That question seems to hinge on the intricacies of copyright law, but as I've stated before, I think the fair use argument is quite strong.
For now. But when you can generate work in the style of an artist without paying that artist, I would. More importantly, media production companies will and so will use access to people's post work to put them out of work now.
We need to distribute the benefits, we're drifting towards copyright infringement being allowed by rich corporation (with resources to generate "AI") as long as they abstract it through a transformer, or whatever.
You can't copyright a "style".
The 3% number was a conservative rounding of the same calculation, but retaining fp16 rather than quantizing to 4 bits.
Here's my original back of the napkin analysis:
Why jump straight to one of the longest popular books in English? I'm sure plenty of people who would otherwise buy poetry anthologies are asking ChatGPT to write them a haiku. It's also entirely conceivable that ChatGPT can perfectly reproduce a poem in its training set - it can produce paragraph-sized quotes.
https://apnews.com/article/openai-lawsuit-authors-grisham-ge...
Not by the person who feeds data into the model... and not by the model itself, for merely retaining an unrecognizable, unusable, highly-compressed and -transformed representation internally.
Seems simple enough. Any other outcome will either halt the process of human intellectual evolution, or transfer a staggering amount of power to people who DGAF about US copyright law. So be careful what you wish for.
If someone studies every one of Stephen King's books, studies King's style, studies his plots, etc. and uses the knowledge to learn how to be a writer, there is no need to invoke "fair use" unless they are actually copying King's books. Studying an author's corpus in order to be able to produce works like the author is not copyright infringement. It is just the way we learn...and the way LLMs learn as well.
If we want to say it isn't allowed because LLM read too quickly, then we could do that. Maybe make them not read any faster than humans. But LLMs aren't doing something different than humans other than just the speed at which they do it.
They are literally arguing they should be allowed to illegally download stuff so long as no human reads the downloaded work directly.
To further your analogy, it would be like your prospective writer objecting to being arrested for stealing the books from the bookstore because he just wanted to become a better writer by studying them, not read them for entertainment purposes.
The copyright infringement in Google's case was not illegally obtaining the original work, it was about the snippets they provided.
In OpenAI's case, it is also about the illegal obtainment (downloading) of the original work, which they admit to in their brief, but argue shouldn't be illegal if they intend only to transform it. That element was not present in AGI v Google.
> Perhaps the most compelling case on point is Authors Guild v. Google. There, search engine Google digitally scanned “tens of millions of books” without their authors’ permission for inclusion in a searchable digital database.
If I break into the court house to let my child see the artwork in the public areas, the issue isn't whether my child can see, learn from, and copy the style of the artwork. The issue is that I broke into the court house instead of going in the front door when it was open to the public.
Did you know it's actually illegal to photocopy books you check out from the library? It's an illegal copy.
That aside, the analogies between humans and ML models fall apart when you actually get into the details, which is why AI companies don't want that.
Human brains don't have RAM and Disk storage that keep direct copies of works. Computers do. Human brains are not entirely deterministic calculators. Computers are.
If you have all the state data of an AI model, you can know exactly what it will produce, and the input data is a necessary component of that state data, so it's not transformative in the same way as a human brain in the first place.
2 humans will not create the exact same output given a shared starting state in the way that 2 computers (AI models included) will.
The only reason people falsely believe AI models are different is because no one actually traces through that state data, and there are enough pseudo-random variables involved and state-changes that they create the illusion of non-determinism, but that is, in point of fact, an illusion.
But some LLMs seem to be able to quote on demand from a wide range of literature; these models evidently do have some kind of memory, even if it's been distilled and blended and distilled again.
> By analyzing large corpora (which necessarily involves first making copies of the data to be analyzed
Whether they retain that later is immaterial to whether copyright infringement occurred. Deleting a movie you illegally downloaded after you watch (train on) it doesn't make it legal.
You didn't finish the analogy.
Not just produce "works like the author" but being able to regenerate those original works word-for-word.
Is that copyright infringement? If I have the entire Dark Tower series memorized word-for-word, and reproduce passages or entire chapters on-demand for others. It's just the way some people learn.
LLMs that are just spitting out full copies of what they were trained on aren't functioning correctly or haven't been trained on enough content.
So all of them? None of the LLM's are "functioning correctly"? Because I can get every LLM to reproduce passages from a book.
You're not the only commenter that has pointed out to me that the model doesn't contain the training corpus; that the model is analogous to a human writer who learns to write by becoming a voracious reader.
I acknowledge that, I think the analogy is fair. The problem is that the memory capacity of the model appears to be humungous; I've never played with ChatGPT, but I understand that it can quote accurately and at some length from a wide range of literature.
By the model, I mean the model behind ChatGPT. I suspect this "memory" amounts to having unimaginable numbers of parameters, i.e. something like overfit. I realize that's just one LLM, and there are other models that are neither generative nor language models, and that have to be much more restrained in what they consider a "parameter".