https://github.blog/2015-03-09-open-source-license-usage-on-...
So we either 1) declare this to not be copyright infringement, or 2) basically outright outlaw training of interesting ML models.
I personally believe that (1) is the way to go, and I find the whole outrage about Copilot to be essentially akin to collectively shooting ourselves in the foot in the long run.
I think we shouldn't forget why copyright exists in the first place. To quote the copyright clause of the US constitution (emphasis mine):
> "*To promote the Progress of Science and useful Arts*, by securing for limited Times to Authors and Inventors the exclusive Right to their respective Writings and Discoveries."
If anything we already went too far with how draconian modern copyright enforcement is, up to the point that it often actively hampers our progress. Please don't make it even worse!
- We could extend copyright statute to explicitly allow training ML models only under particular conditions (for example, some kind of "non-commercial use" clause).
- We could enact mechanical licensing for inputs to ML models, as exists for musical cover performances. Creators of the works used as inputs would get royalties.
- We could require asking for permission to use a work as input, but standardized generous terms once permission is granted.
- We could flip that and just require notice that you're about to use a work as input, but allow the creator a certain window to object.
- We could declare that inputs are a free-for-all, automatically non-infringing, but that you (the creator of the model) are liable for extra damages if the outputs are proven to be able to infringe.
Many, many different ways this could shake out. I'm not particularly optimistic that any of (what I see as) the better choices will come to pass, but they exist.
Saying that it’s infringement doesn’t mean we’re outlawing interesting ML. These definitely aren’t the only two options. A few years ago, this kind of infringement wasn’t even possible, and now it’s just beginning. This stuff is in its infancy.
Reliability and controllability is an issue our field has to confront as applications move from research to application. This is just another instance of that. We’re all eager to deploy this stuff as soon as it can do something cool, but that’s not enough, as all the discussion around copilot demonstrates.
> I think we shouldn't forget why copyright exists in the first place… To promote the Progress of Science and useful Arts
Right. It’s to encourage creators to create things. I wrote about this elsewhere [0] that the issue of generative ML models is that, as far as incentives go, you can pay a creator for use of their things, or you can throw money at AWS and get use of something like that creator’s things. The fair use equation is ‘$$$ + my stuff = your stuff,’ except that that money doesn’t go to the creators, it goes to nvidia and aws.
I think we’ll need new copyright laws for this, to make a middle ground. I personally want to be able to exclude my works from being training data unless I license then openly enough. Just like with open code licenses where you opt in. That option doesn’t seem to exist right now.
Wouldn’t it be great if a model benefiting from all this public data had to be public itself, or something along those lines? Wouldn’t that promote the arts and sciences even more?
Would you dramatically expand what you can legally borrow without infringing or just allow wholesale automated washing from one copyright to another?
What do you do when random corp starts shutting down open source over ownership of code it swiped from project claiming infringement?
You're acting like we don't already have something similar. We do. It's called fair use. (: I'd probably just expand on that. If the portion of the work generated by a model is insubstantial, and/or doesn't harm the interests of the original rights holder then it should be okay.
If your model can generate a single excerpt of Harry Potter does that constitute harm to J.K. Rowling? Can you use it to reasonably generate a substantial portion of the book with it? Is it a reasonable substitute for buying the book? No? Then it's okay.
Some countries in fact do already have such exceptions, e.g. Japan amended its copyright law in 2018 to explicitly allow for this. To quote the relevant law:
> "It is permissible to exploit work in any way and to the extent considered necessary ...where such exploitation is not for enjoying or causing another person to enjoy the ideas or emotions expressed in such work..."
So if you want to train an ML model on a bunch of commercial novels, without the permission of their owners, and then exploit that commercially - you can do it, and it's completely legal.
Personally, I'd like not have my IP used in training ML models. If they want to incorporate my stuff into their training data, they can do it on my terms, which would be either paying to license my work, or something along the lines of a new kind of copyleft license where the model has to be public and copyleft too. By including my work anyways, I didn't get the terms I would have needed.
Scale makes it weird, because it's the a-drop-of-water-in-an-ocean issue, but harm is still real harm when it's split into a billion separate pieces of infinitesimal harm.
In other words, to what extent does the color of the bits matter? [1] And does the output of a ML model have a new "color"?
I have been effectively using GPT-3 a lot recently because I added chapters with OpenAI APIs examples to my Common Lisp and Clojure books (you can get free copies at [1] by setting the price to Free).
At Capital One, I used LSTM models to generate AWS JSON logs for testing purposes. The original logs contained sensitive data that was not in the generated data. JSON structure and names of keys was learned and reproduced verbatim, but not the values. GPT-3 has massively more parameters than my models. BTW, I also generated spreadsheet data using GANs: similar idea, but different tech.
CHAPTER ONE
THE BOY WHO LIVED
CHAPTER ONE
THE BOY WHO LIVED
Mr.