News for hackers. Stuff that matters.
Re:OpenAI's justification for why training data is fair use,... by troupe on Thursday October 05, 2023 @02:49PM (#37782482)
If you are saying that they illegally obtained King's book, then the issue is the fact that the stole the book from somewhere. I don't disagree with your assessment there. But if they checked the book out from the library and let the LLM read it, the fact that it has read it shouldn't be any different than if you had read it as a human.

If I break into the court house to let my child see the artwork in the public areas, the issue isn't whether my child can see, learn from, and copy the style of the artwork. The issue is that I broke into the court house instead of going in the front door when it was open to the public.

Re:OpenAI's justification for why training data is fair use,... by ang_cire on Thursday October 05, 2023 @03:17PM (#37782972)
> if they checked the book out from the library and let the LLM read it

Did you know it's actually illegal to photocopy books you check out from the library? It's an illegal copy.

That aside, the analogies between humans and ML models fall apart when you actually get into the details, which is why AI companies don't want that.

Human brains don't have RAM and Disk storage that keep direct copies of works. Computers do. Human brains are not entirely deterministic calculators. Computers are.

If you have all the state data of an AI model, you can know exactly what it will produce, and the input data is a necessary component of that state data, so it's not transformative in the same way as a human brain in the first place.

2 humans will not create the exact same output given a shared starting state in the way that 2 computers (AI models included) will.

The only reason people falsely believe AI models are different is because no one actually traces through that state data, and there are enough pseudo-random variables involved and state-changes that they create the illusion of non-determinism, but that is, in point of fact, an illusion.

Re:OpenAI's justification for why training data is fair use,... by troupe on Thursday October 05, 2023 @03:38PM (#37783273)
You are assuming that an LLM is saving a copy of the book it reads.
Re:OpenAI's justification for why training data is fair use,... by pclmulqdq on Thursday October 05, 2023 @05:24PM (#37784477)
LLMs don't read, they are trained. Yes, training involves saving a copy of the training corpus.
Re:OpenAI's justification for why training data is fair use,... by denton-scratch on Thursday October 05, 2023 @05:59PM (#37784811)
The LLM is the output of the training system; the LLM doesn't do the reading, the trainer does. The LLM doesn't "contain" a copy of the book.

But some LLMs seem to be able to quote on demand from a wide range of literature; these models evidently do have some kind of memory, even if it's been distilled and blended and distilled again.

Re:OpenAI's justification for why training data is fair use,... by ang_cire on Friday October 06, 2023 @04:33PM (#37796014)
No, I am actually reading OpenAI's PDF linked here, where they say they make a copy of the online corpa to train on, right at the top of page 2:

> By analyzing large corpora (which necessarily involves first making copies of the data to be analyzed

Whether they retain that later is immaterial to whether copyright infringement occurred. Deleting a movie you illegally downloaded after you watch (train on) it doesn't make it legal.

Re:OpenAI's justification for why training data is fair use,... by troupe on Saturday October 07, 2023 @03:22PM (#37804682)
I agree with you. What I'm trying to differentiate is between the way you obtain a book and whether training an LLM on a book that you own the rights to read should be considered copyright infringement.
Re:OpenAI's justification for why training data is fair use,... by ang_cire on Monday October 09, 2023 @02:20AM (#37817460)
Unless the license you have for the book includes rights for commercial usage, it should be illegal to train a model that is being used for business, imo. In terms of individuals training models for their own personal use, I think that becomes much more tough to answer, but that's not really cropping up in the legal fights.