News for hackers. Stuff that matters.
Re:The Pile is a 825 GiB diverse, open source language model... by IshKebab on Thursday March 07, 2024 @02:59PM (#39633905)
Yeah I agree. If 2 contains complete copyright works (e.g. all of Harry Potter) then "we're just using it for AI training!" stands approximately zero chance of passing the fair use test. Their assertion that it does is just wishful thinking.
Re:The Pile is a 825 GiB diverse, open source language model... by HenryBemis on Thursday March 07, 2024 @03:44PM (#39634375)
I'm playing stupid now: I believe that if I ask the LLM to "display Harry Potter Book 1" and it does, word-by-word, then you're 100% right, it's copyright infringement. But, if I ask the LLM to "give me an analysis of the "Professor Severus Snape's" character and it gives me one, then I don't see the problem.

So in that sense I understand the response that "they don't violate copyright" by studying the material. Again, I don't pretend to be a lawyer, and not every law has to follow my logic.

Re:The Pile is a 825 GiB diverse, open source language model... by swatcoder on Thursday March 07, 2024 @04:02PM (#39634537)
That's a different discussion.

This isn't about the output for content generators or about the abstract numeric weights that they operate over. That's more complex and a largely open question.

But this is literally about indiscriminately distributing copyrighted works in a large, convenient archive while arguing that it's okay because you normalized the formatting a bit and because you suspect that some people might find "fair use" value in it.

Re:The Pile is a 825 GiB diverse, open source language model... by fennecfoxy on Friday March 08, 2024 @06:03AM (#39639943)
This will probably get buried, but a lot of the GOTCHA! LLM copyright bullshit is "it produces this passage from x book or y article perfectly!"

Meanwhile the complainant (such as GRRM) forgets that often passages from said articles and books are strewn throughout the Internet. Of course chat GPT can drop passages from the GoT books; there's several entire fucking wikis for that franchise that reference passages, quotes, details etc.

Same goes for news articles, passages of which are often quoted by other sources or websites.

Not that chatGPT has reproduced many works in whole, but it's an interesting logic problem for fair use law: if I have copyrighted article X, but websites ABCDEF all quote various passages of my article (ie fair use, critique, etc) and then ABCDEF is used to train an LLM, if the LLM can _reassemble_ the article from quoted passages without referencing article X itself, is it copyright infringement or fair use?

Re:The Pile is a 825 GiB diverse, open source language model... by whimsicalism on Friday March 08, 2024 @10:36AM (#39642004)
I think it is almost certain that GPT was trained on full books
Re:The Pile is a 825 GiB diverse, open source language model... by mistrial9 on Thursday March 07, 2024 @03:40PM (#39634338)
said with confidence, however Silverman et al Judge explicitly rejected what you just asserted AFAIK
Re:The Pile is a 825 GiB diverse, open source language model... by papercrane on Thursday March 07, 2024 @03:50PM (#39634435)
You've got it backwards. The judge in Silverman et al dismissed the claims asserting that OpenAIs output is copyright infringement. The claims for copyright infringement in the training data are still going forward, that will directly test whether it is "fair use" or not.

From the ruling:

> Assuming the truth of Plaintiffs’ allegations - that Defendants used Plaintiffs’ copyrighted works to train their language models for commercial profit - the Court concludes that Defendants’ conduct may constitute an unfair practice.6 Therefore, this portion of the UCL claim may proceed.

https://caselaw.findlaw.com/court/us-dis-crt-n-d-cal/1158180...

Re:The Pile is a 825 GiB diverse, open source language model... by mistrial9 on Thursday March 07, 2024 @04:59PM (#39635072)
aha - much appreciated
Re:The Pile is a 825 GiB diverse, open source language model... by whimsicalism on Thursday March 07, 2024 @03:44PM (#39634377)
no, she didn’t reject the claim that training on copyrighted work is infringement, merely that the outputs are not infringing simply by bearing similarity to the texts