I think if anything, this helps to demonstrate the absurdity of ancient copyright law in our modern world. I would rather move in the direction of abolishing copyright rather than in the direction of strengthening it.
As do I, but this is not expanding copyright. This is saying that as long as copyright exists, AI model training has to respect it too, and can't be a laundering operation for license violations. Otherwise, you're giving companies building AI extra permission to ignore Open Source licenses and small copyright holders (e.g. independent artists, authors of stories/comments/text on the Internet), which is an asymmetry in favor of companies with the resources to train huge models.
I'm all for abolishing copyright, and I think it is absurd. What I'm against here is the asymmetry of keeping all the harms of copyright around while letting AI training violate it.
From my experience this is incorrect; it's not an asymmetry in favor of companies; if anything it's an asymmetry in favor of small players.
How do I know? Because I myself am one of those small players, and because I train machine learning models myself. Not huge ones of course, but ones which need data in the range of e.g. 10~25GB range. As an individual it would be completely impossible to explicitly license this data for training (most of which is owned, mind you, by huge corporations!), while a corporation wouldn't have a problem with it.
> What I'm against here is the asymmetry of keeping all the harms of copyright around while letting AI training violate it.
This is fair enough. But please beware that the blast radius of entities your proposal would harm is not limited to huge corporations. I would prefer to fundamentally reduce the scope of copyright too instead of carving out special cases for AI training, but we both know that isn't going to happen.
I'm well aware that this is widely done by large and small entities alike; I'm not just concerned about the practices of large companies, I'm also concerned about small ones, and individuals.
This is not "more draconian" copyright; this is not a change to copyright at all. This is the same copyright we already have, equally enforced for all copyright holders, large and small. You want to "democratize" this space? Get rid of copyright, and a lot of things become better and easier, not just AI.
I don't want the dystopia where copyright still exists for large publishers and studios and proprietary software companies to prevent sharing and remixing things, but at the same time all the small and Open Source entities don't get to set their own terms because AI will just remix them away.
Do you think you'd get away with training an AI on a pile of Disney and Pixar movies? Or is it just the myriad of smaller copyright holders, like independent artists, that you're comfortable stepping on?
> It's not going to stop OpenAI. It's not going to stop Microsoft. They'll figure it out.
Will they? If there isn't, in fact, a legal solution, they're not in any better shape than anyone else. If anything, they're in a worse position, because they have deep pockets for potential lawsuits, while non-commercial efforts tend to not be interesting targets to sue (at most, they get shut down, and others pop up in their place).
Question your assumptions about the world that results from requiring AI to respect Open Source licensing and other small copyright holders such as independent artists or online comment/story authors. It's not a corporate dystopia. It's a level playing field.
Maybe it's not evident to non-practitioners in the field, but to every serious practitioner it is obvious that you can't train a state-of-art model without a lot of data (at least right now without some colossal breakthrough), and that actually licensing that data is not really practical (because you need terabytes of it), and it's definitely going to be impossible for anyone who isn't a megacorporation.
Can we agree on this point? If not can you please explain how do you think that e.g. a single individual like me will be able to train e.g. an image diffusion model (so I'd need a few terabytes of images) if I have to respect the licenses of every image in the training set?
Okay, so I hope we can agree that it won't be possible? So now here's the question: do we want such AI models to exist, or do we want to make them illegal (and maybe available only to huge megacorporations)? These are our only two choices, which logically follow from the requirement that we need a lot of data for training.
What I'm advocating for is that we should allow such models and that they're beneficial to us as a society, hence the "and we must be able to do this" in my argument.
I'm starting with the assumption that I want these models to exists and that everyone should have access to them, and then go backwards from that. What you're starting with is the assumption that the training data copyright should be respected, and you're going backwards from that. But these two graphs are not connected, which is why we can't agree.
Or in other words, what you're (indirectly) advocating for is to make those large models effectively illegal. This is, of course, a valid stance, and if you want to take it then you're free to do so. But that's objectively what you're proposing in practice, and personally I disagree with it.
> Do you think you'd get away with training an AI on a bunch of animated Disney movies, and asking it to generate new images in that style, and using the result in commercial endeavors?
Yes.
Just the same as if I'd draw an image in the style of an animated Disney movie by hand.
In both cases I'll be sued for trademark infringement if the image's of the Mickey Mouse though.
In many cases the current "inequality" of how law is applied to individuals and to megacorporations has little to do with the law itself, and everything to do with how rich the megacorporation is. Try to set up an apple orchard and pick an apple as a logo[1] and tell me how it goes. The law explicitly states that another company, say one which produces computers instead of actual apples, has no merit here, but alas they have deep pockets, so here we are.
[1]: https://www.wired.co.uk/article/apple-vs-apples-trademark-ba...
> Question your assumptions about the world that results from requiring AI to respect Open Source licensing and other small copyright holders such as independent artists or online comment/story authors. It's not a corporate dystopia. It's a level playing field.
Well, let's see, for the sake of argument let's assume that the current widely believed legal status quo is true. (That is, that you can train a model on any data regardless of copyright because it's fair use. Although in my country that's explicitly allowed by law so here we don't have to assume anything.) Right now OpenAI can scrape 1TB of data off the Internet and legally train a model. I can also scrape 1TB of data off the Internet and legally train a model. And it can be any data, not just open source programs and content produced by small copyright holders. Is this not a level playing field?
Are you seriously suggesting that having to pay billions of dollars to license the training data necessary to train a model is a level playing field? I guess if nobody will be able to do it then it will be, in a way, a level playing field; I just fear that entities with enough money will be able to license enough data anyway and then the rest of us will end up with nothing.
Yes. Creating a new animation that is inspired by Disney's style is clearly fair use. Otherwise Studio Ghibli would be in trouble.
I think this is an incredibly short sighted take.
Training a modern state-of-art LLM needs terabytes of training data. It's probably not going to be practically possible to actually license this much data. And even if we assume it will be then the only entities which will be able to do it are the world's biggest corporations.
So if AI models are a derivative work of their training data then as a consequence the whole field will mostly die overnight, with perhaps only a few of the world's richest corporations being able to play with this space.
I'm sure you're aware of the slew of really cool open/free models that are out there, which you can download today and play with on your local machine. Like Stable Diffusion. It was trained on all rights reserved data. Now it's going to be illegal. Or OpenLLaMA/Falcon/MPT/etc. Also trained on all rights reserved data. Illegal too.
Have you heard about the Pile dataset[1]? It's the most popular open dataset for training LLMs, and essentially every non-proprietary LLM is trained on it, or on parts of it. Do you know that it contains 100GB of all-rights-reserved pirated ebooks? If AI models are a derivative work of their training data then all of those models are now illegal.
[1] - https://pile.eleuther.ai
This would completely kill any chance of having good free/open models. Sure, you could then grab all of GPL'd code, train a model on that, and maybe have a decent working GPL'd model that can emit GPL'd code. There might be enough data for that. But that's it. What about other kinds of models? Image generation, chat bots, personal assistants, story writers, etc. There's just not enough freely licensed data (and probably will never be) to train those.
We must democratize this space. It's already insanely expensive to train state-of-art LLMs; we don't need to make it even more expensive. It's not going to stop OpenAI. It's not going to stop Microsoft. They'll figure it out. What it'll stop is everyone else, and will make this technology completely out of reach for everyone who isn't an insanely rich multinational corporation.
With all due respect, to everyone who's complaining about Microsoft using your code to train Codex, I'll be blunt: you're advocating for collectively shooting all of us in the foot. I don't want to live in a dystopia where only huge corporations will have access to cutting edge AI technology, so please stop pushing in that direction by advocating for more draconian copyright just because you're butthurt that Microsoft/OpenAI used your code as training data without asking for permission.
Big corporations are not the only people doing this; the little guys (e.g. EleutherAI) who release free and open source models do it too. If you try to block the big bad evil corporation from doing it you'll also block the little guys. The major difference here is that the little guys will get completely screwed by this while the corporation has the cash to try to get around it.