> “You should know that training on millions of books is not the only way GPT-3 learns. It is just the foundation of its knowledge.”
I’m not a lawyer, but to me it seems within the realm of possibility that a U.S. court eventually finds strongly in favor of the copyright holders, the Supreme Court agrees (because Big Tech has so few friends left), and OpenAI will be required to destroy the GPT-3 model and all copies of the training data because they can’t filter out copyrighted works.
Re:Is Copyright Eating AI?
by notahacker on Sunday January 22, 2023 @09:47AM
(#34477836)
Yeah, I think "we can't actually tell which bits of our model are derived from your work which we copied without your permission onto our system for training purposes probably makes AI companies more vulnerable rather than less. Other platforms like search and social media have successfully defended themselves by showing willingness to promptly remove copyrighted material, let copyright holders opt out of indexing or even negotiating stuff like ContentID so copyright holders get paid each time their work is used.
Re:Is Copyright Eating AI?
by FartyMcFarter on Sunday January 22, 2023 @04:44AM
(#34475822)
Your reply is mostly a red herring because as you yourself said, the OP was talking about regular training data, not the ICL you focused on.
Just because you can find one way that GPT might be slow doesn't invalidate the point that its training does use massive amounts of data.
Re:Is Copyright Eating AI?
by i_like_apis on Sunday January 22, 2023 @06:05AM
(#34476216)
So what if it does?
If GPT-3 can learn in context it means both the training set and the prompt could be in copyright violation. So even a clean model, trained on licensed data, cannot guarantee there will be no copyright issue.
But I think you meant GPT-3 has seen many books during training, not during inference. You should know that training on millions of books is not the only way GPT-3 learns. It is just the foundation of its knowledge.
GPT-3 learns "in-context", that means it can learn a new word or a new task at first sight. It just needs a description or a few examples. This is the most powerful feature of GPT-3 - in-context learning. And when it comes to ICL, it is much like humans - only sees a few examples, not millions of books.
> “Do you ask for permission when you train your mind on copyrighted books?”
The nature of ICL is that it happens at prediction time. So GPT-3 would have to explicitly be instructed to learn a specific skill. Should it reject instructions if they are sourced from copyrighted books?