This also important for Source Available (for your eyes only) licenses. People don't care about them either.
This whole AI thing is a boundless ethics violation in every category, yet people don't care, because "it's awesome!" for reasons I don't understand.
Addendum: The reason why OpenAI has a non-profit part is to allow them to claim that they do non-profit research, which fair-use doctrine requires to work.
1. "Fair use"
It is fair use to use snippets of my code in other projects. AGPL doesn't change that. If I have a 3-line sort routine, and Copilot copies that verbatim, it's okay.
The place and reason Copilot is not okay is because I can ask it to spit out code which, in some versions, was almost verbatim code I wrote, and in others, a paraphrase. The same abstractions and high-level design I invented is copied by GPT.
It's like writing a book set in the Star Trek universe and (not as a parody), changing "Spock" to "Spork" and "Kirk" to "Dirk."
It's a problem for me because I compete with commercial systems, and chose the AGPL for very specific reasons: People have a right to know how their data is analyzed, and AGPL preserves that. Copilot allows companies to quickly replicate and whitewash my code.
If Microsoft uses a snippet in Windows -- e.g. a small use in an unrelated context, fair use clearly allows that. As much as I don't want my code stolen, degrading fair use would be the far greater evil.
2. "The reason why OpenAI has a non-profit part is to allow them to claim that they do non-profit research, which fair-use doctrine requires to work."
No, OpenAI was set up as a genuine, if ill-formed, non-profit. When it clearly became big $$$, bad people (likely breaking the law) formed a for-profit arm to personally profit from it. The non-profit arm is there for historical reasons.
Your statement is false, and there is ample historical record that what you're stating is not how it happened.
That's dangerous, since the wrong lessons would be drawn. The lesson this reiterates, which I learned many times the hard way, is that for non-profits, checks-and-balances matter. If you don't want your non-profit going south, this means everything -- licenses, corporate charter, bylaws, contracts, etc. -- need to be properly structured to act as a permanent network of checks-and-balances so the organization cannot go south even if bad people are involved, and that incentives are structured to support good behavior.
We do need a for-profit like the original OpenAI, but properly structured and resourced.
That depends. You may have a 30 line function which can be paper-worthy. I'm not supporting software patents, but reasonable attribution. If somebody uses my GPL licensed novel algorithm (which is a single function) to get unfair commercial advantage and doesn't adhere to the GPL license, can we say it's fair use? Example? See [0].
> It's a problem for me because I compete with commercial systems, and chose the AGPL for very specific reasons: People have a right to know how their data is analyzed, and AGPL preserves that. Copilot allows companies to quickly replicate and whitewash my code.
We're on the same page. I don't want my algorithms to be used by people verbatim w/o attribution or without proper license. Don't like the license? Reimplement it, paper is there.
> No, OpenAI was set up as a genuine, if ill-formed, non-profit. ... [snipped for brevity]
You're right. Let me rewrite my statement: "The reason why OpenAI keeps a non-profit part and has this convoluted many-company structure is to allow them to claim that they do non-profit research, which fair-use doctrine requires to work."
Hope that it's better now.
> Your statement is false, and there is ample historical record that what you're stating is not how it happened.
About OpenAI's structure, or GPL being fed to CoPilot? I didn't say anything about OpenAI's history AFAIK, but I have seen a tweet, containing a snapshot of an e-mail saying that "Copilot is indeed trained on every public repository on GitHub". The tweet is deleted later probably of threats. Just because of this tweet, I'm keeping this list [1].
Actually we're literally on the same page. Only difference is I keep my "plagiarism" threshold at "function" level, because they can contain some bigger magic than themselves, and you keep it at "design" level where the magic happens as a collection of functions + design.
This is probably due to different domains we operate in, in mine, a single function can affect accuracy and speed so much that it can be considered secret sauce. Welcome to HPC / Scientific Computing. :)
> We do need a for-profit like the original OpenAI, but properly structured and resourced.
When you do for-profit, fair use goes out of window. See [2].
[0]: https://journals.tubitak.gov.tr/elektrik/vol29/iss2/45/
[1]: https://notes.bayindirh.io/notes/Lists/Discussions+about+Art...
> That depends. You may have a 30 line function which can be paper-worthy
We have no disagreement. That's why I gave a pretty generic 3-line function. 90% of the upside of ML tools for me, by the way, is generating very routine code, so there's not even too much friction yet. I can prototype a parser with e.g. 5 XML libraries I've never used in 15 minutes, which means I make better design choices.
> You're right. Let me rewrite my statement: "The reason why OpenAI keeps a non-profit part and has this convoluted many-company structure is to allow them to claim that they do non-profit research, which fair-use doctrine requires to work."
> Hope that it's better now.
Nope. It's not. The reason the non-profit part is around is right now, they're skirting the law. If it went away, they'd be very clearly committing fraud.
The non-profit received support from donors -- including from the government in the form of a tax waiver and the general public in the forms of non-financial support (such as positive buzz / articles / scientific work) -- based on a 501(c)3 registration and a belief it would act in the public interest.
The instant the 501(c)3 part went away, this would all be fraud. Everyone who helped make OpenAI successful, from the grad student who spoke about it glowingly at a conference to the federal government, would have been embezzled.
I feel bad about having supported them, and I bet others do too.
It is extremely worrying.
Being non-profit, or non-commerciality in general, has relatively little to do with fair use.
NYTimes is suing OpenAI because it siphoned paid articles and reproduced in verbatim.
That's basically the same thing it does to open source code, so if NYTimes wins, there might be some hope.
The same way an LLM would generate that code without the license, another LLM could find that code and determine that it infringes. If anything, the emergence of LLM powered agents will empower the free software movement to find infringements.
The only way free software would "end", is if judges would not be able to enforce copyright, because for example the copyright infringement is obfuscated too much. For example if LLM powered agents are going to both generate and run the code without ever saving it to disk. We're not quite there yet.
We have entire professions (musicians, actors) who develop this ability.
Is that, in your view, 'illegitimate' too?
Why or why not?
Also, comparing a human to a service that runs on billions of dollars of infrastructure is silly.
I have AI LMs running locally on my GPU. They cost less than a photocopier or CD burner to operate. Pennies.
I'm sure you've heard of covers? Well every cover that is published affords a royalty to (at least) the original authors of both the lyrics and composition. The artist may get some money, depending on how the work was licensed.
Do you see a difference between training and reproduction?
Yes someone may teach another person based on their memory, but even if that person still performs that work they still are legally required to license the work.
Yes, if the only thing they were doing was training, sure. But it’s not. They’re training and then presenting the data and given the way LLMs are trained, there is no guarantee a transformation even takes place.
At the end of the day LLMs should be licensed under the current copyright system. Maybe OpenAI need to donate some money to a few politicians for that to change.
https://www.youtube.com/watch?v=MAFUdIZnI5o
It doesn't really matter what my or anyone's view on it is, unless they're either a law maker or a judge.
People do not get sued for having abilities. They get sued for publishing copyrighted works. What tools they use to create or publish those works is not relevant.
The only interesting legal question in my opinion is wether the large language model itself can be considered a carrier of copyrighted information, like for example an encrypted DVD would be.
If you can come up with a prompt that itself does not contain substantial copyrighted information, and that prompt leads the model to come up with a substantial amount of copyrighted information, then in my opinion it's no different from an encrypted DVD, and as such would illegal to distribute without licensing. Also I believe it would be illegal to provide access to it over the web like OpenAI does.
IANAL, but I wouldn't hold stock in any company that publishes (access to) models that are easily goaded into producing substantial copyrighted information.
In fact, I once asked an actual lawyer about the information-theoretic questions raised by illegal/infringing content known to be on an encrypted storage device.
Her answer was "Yeahhhh, no, the Judge is probably not going to see it that way." :-)
However, copyright law seems to make a distinction between expression/implementation of an idea (which can be copyrighted), and the idea itself (which generally speaking, cannot be). This is why clean room implementation is a legally sanctioned protocol to copy functionality without running afoul of copyright law.
Whether the language model is copying the "expression" (or a particular implementation) of a computating function, or whether it is just somehow re-implementing it given user's instructions, is going to be an interesting debate, and I really doubt people who think it's one way or the other has actually understood the nuances.
LLMs/Copilot/etc. copying other people's code is not legitimate for precisely the same reason. But big and powerful beats small and week.
To translate that to LLMs: training an LLM on songs (or code) is fine. It's the output and what you do with that output that might be problematic.
Only if you assume that LLMs are treated like people. Does an LLM have a right for decent working conditions? Is it illegal to make an LLM work 24h a day?
LLMs are not humans, therefore it seems reasonable to not blindly translate everything that applies to humans.
One problem I see with LLMs is that it is killing the job of many artists by using the copyrighted work of those very artists. The more the artists work, e.g. by doing truly original content that an LLM cannot generate because it was not trained on that, the better the LLMs get at killing the job of the artists.
It is a very weird situation: the LLMs need the artists in order to finish killing the artists. When that is finally done, it's not clear at all that LLMs will be able to generate new truly original content or whether they will be stuck in the world that the artists created before dying.
It's very different from, say, a calculator that replaced the job that real person had. Because the calculator can do at least as well as the human was doing, and the calculator was not feeding from the output of those humans.
Are you old enough to remember when humans planned in advance which video you would watch next?
Well if it prevents humans from producing art, of course this will happen. Doesn't mean it will be better art. I would argue that it will be worse by definition.
Very much so. But now that time is back.
Most people I know do exactly that. The time when everyone consumed TV passively is over.
https://www.npr.org/2015/09/23/442761620/after-copyright-rul...
https://variety.com/lists/song-copyright-infringement-cases-...
I don't personally think running copyright material through a magic box that you write lots of pretty marketing about removes copyright. Much like running it through grep to delete the copyright notices doesn't remove the copyright. Or deleting the licence file from a fork.
Seems totally obvious to me, but only because I don't buy that there's an artificial intelligence composing code from scratch that just happens to have the same expletives in comments that the GPL code did.
Your point is obvious to everyone. The question is why do you think anyone else believes differently?
So either they believe it isn't a copyright erasure tool, or they believe it is and don't care, or they haven't formed a belief in either direction.
That case analysis looks complete to me.
All of these are GPL violations if it can be shown you looked at a GPL function while copying this.
On the other hand, if you came up with it on your own and it happened to look similar, then it's fine.
If there is a threshold, then it is possible to rewrite the original code in ways that do not resemble the original code verbatim, but are effectively the same.
That's an interesting question. In general, the answer is no, but could change depending on if you had the function open at the time of writing the new function.
But a lot of us have knowledge of basic data structures, etc, from reading GPL-licensed code (or perhaps even proprietary code in textbooks). So, in general, coming up with your own understanding is fine.
The violation is where you subsequently pass it off as a novel creation.
Being hard to solve doesn't make a problem go away, though. This is a place where I'm somewhat curious where it ends up. Frankly I think the outcome is more interesting for artistic works (e.g. poetry or images) than practical ones like code, since current generations of AI seem so much better at the former. Still interesting either way.
Why would you think this is 'legitimate'? Firstly, LLMs are not legal agents, only humans are, so any human caught doing this is violating the GPL, whether or not the creation of the LLM is legal or not. Bits have color
They say "IF it is legitimate to do X, then ...". They don't say it is legitimate.
If copyright is dead, then no need for licenses anymore: anyway people can remove your copyright by laundering your code through an LLM.
But it's not legitimate. That's what I said. If you (a human) take copyrighted code and redistribute it against the terms of the license, you are guilty of a crime in most places.
That was my point, whether the language model was built legally or not, if you have control of such a language model and use it to produce copyrighted code and then distribute that, you will be held as violating copyright.
I think there's a lot of confusion here because some on the thread seem to believe a large language model can in and of itself be guilty of a crime. It can't. The trainer may be guilty, or the user could be guilty, or both could be guilty. The model simply is. It might not be redistributable depending on how courts interpret the training procedure (remains to be seen).
I don't really care who we should theoretically punish: once it's out there, it's out there for good. Maybe it's already too late, and the LLM people just broke copyright for everybody (or worse: they broke it for honest people). So much for making the world a better place.
"if a third party sues a commercial customer for copyright infringement for using Microsoft’s Copilots or the output they generate, we will defend the customer and pay the amount of any adverse judgments or settlements that result from the lawsuit, as long as the customer used the guardrails and content filters we have built into our products."
https://blogs.microsoft.com/on-the-issues/2023/09/07/copilot...
At the very least it's somewhat ironic. At worst it's plain hypocrite.
If I copy-paste your code, can I say that my computer "read your code, learned from it and later wrote something inspired and based on the knowledge it internalized"? If I take a photography of a photography, can I sell it as mine, given that my camera "observed the original and learned from it"?
My opinion is that if you read my code, learn from it and later write something inspired and based on the knowledge you internalized, then that is fine. Because you are a human. But if you do anything that it functionally an obfuscated copy-paste and pretend that you actually created that, then that is not fine.
My argument is "LLMs are not humans, therefore we should be careful with such comparisons". Do you also think that LLMs should have rights? Should it be illegal to make an LLM work 24h a day?
> I work in ML, although not LLM, and I do understand how they work.
I never meant to underestimate ML. However it feels like you underestimate humans :-).
> I don't think you can reduce LLMs to obfuscating copy-pasters. I believe they are closer to our internalizing process
Wow. If you think that LLMs are closer to humans than to copy-pasters... you must not think much of living organisms. Now I understand why ML people think that AGI is coming soon :).
I'm saying that the arguments that define this "abuse" often are.
What I specifically find hypocritical, we cannot suddenly put restrictions on what clients may or may not access, read, store and see our content. "Only a human may access my content" is just as bad as these numerous attempts of DRM that "hollywood" has tried to force down on the web.
More practically: I have a blog, with a CC-SA licence on my content. If an LLM starts spewing my exact paragraphs and articles without the same license (the SA-part asks this), I'd be pissed and have a right to call it "abuse" and could quite easily take the company behind it to court (If I wanted to).
But when that LLM merely hashes, vectorifies, trains on, and stores copies of my content so that their project can mimic my tone of voice, ideas and other content, so be it: I find it immoral to impose restrictions on what, who, where and how my content is read. Because I find DRM and such immoral.
What if I copy your article by hand? Or run it through an algorithm that translates it into another language. Can I sell it as my own? Or does it have to go through an LLM for people to sell your work as theirs?
Think about this: what if your job was to write articles, and suddenly nobody buys them anymore because they can read them cheaper on another website that serves them after the LLM laundering? Is it moral for that other website to make money out of your work?
> I find it immoral to impose restrictions on what, who, where and how my content is read.
Did you think that through? Do you find it immoral for a book author to sell their book, and to make it illegal for you to sell copies of it? It feels like you only consider your hobby-blog and ignore those who try to make a living by producing content (e.g. a photographer).
I don't.
A judge will have to decide which it is. So far, they're leaning to "it's not a copy machine"
Edit: and yes. I find it immoral when an author has a say in how I use the book after buying it. Whether I whipe my ass with the paper, or display it in a shrine is non of his business. I bought the carrier, it is mine. And under copyright law (EU), I'm free to sell my copy to a second hand bookstore, and that bookstore is free to sell it and make a profit. I understand the nuances of digital carries that can duplicate their contained knowledge much easier than a paper book can, but I find it immoral that the ebook I buy, cannot be read by my wife on her ereader without her buying it as well.
I see an LLM as a machine that takes an input and automatically gives an output. If that input is copyrighted, and the output is essentially a translation of that input, then it is a problem.
BTW using a camera to take a photo of your code, and then use text recognition to render it (with potential mistakes), and then use an algorithm to correct the mistakes, all that is not exactly "something that copies stuff": you will agree that there is a lot happening. Still you would say that it breaches the copyright "because it is exactly the same text". The thing is, "copyright" is not defined as "exactly the same text" (otherwise you could rename a few variables and be done with it).
> I find it immoral when an author has a say in how I use the book after buying it.
So you find it immoral for an author to sue you because you bought the book, copied it and sold cheaper copies on Amazon? Pretty sure you don't. You can't just ignore all the nuances that don't support your argument... I don't like DRMs either, but it does not mean that the concept of copyright is always wrong.
Group A : "Information wants to be free and we shouldn't patent algorithms"
Group B : "If you want to use our corporate stuff, you must pay"
Group A : "If that is the route you choose, we will still not charge for our stuff but if you want to use it, you must share your changes"
Group B : "If we use this tool that generates code, trained on the code that Group A created, it doesn't count as Group A's code and we can ignore their rules"
Group A : "It does count as our code, please abide by the rules."
More interesting to me would be; If I trained a LLM on leaked MS sources and built an OS or Office suite. Would MS be ok with that?
Why not?
If your implementation is serving back literal chunks of code, I'd presume not, and they can easily use copyright laws to punish you.
If your implementation uses that as "inspiration", It's hardly different from you reading that code and learning from it. Obviously, it's almost impossible to read the code, learn from it, and be 100% sure you'll never reproduce any of their code. Hence "clean room" teams and such.
>Section 102 of the U.S Copyright Act sets forth what is and what is not subject to copyright protection.[3] The first part of the statute states the eight types of works which are subject to copyright protection such as; literary works, musical works, dramatic works, pantomimes and choreographic works, pictorial, graphic, and sculptural works, audiovisual works, sound recordings; and architectural works.[4] The second part of the statute states that copyright protection does not “extend to any idea, procedure, process, system, method of operation, concept, principle, or discovery…”[5] https://easlerlaw.com/software-computer-code-copyrighted
Which is why we have software patents in the first place.
From my limited knowledge, it looks like it is just regurgitating. This is based on what I have heard about outputs, i.e. stays the same if rand function is fixed.
Also with code there are finite ways to create something functional as opposed to an opinion paper where there are a range of viewpoints and how to express those viewpoints.
You can make this argument for any artist. We study whole art movements which are periods of time where all art looks the same. Somehow, we want code production to be completely novel with no prior inspiration?
To your second point. I agree. There is limited expressivity in code implementation. As a human how would you cope with that? You can't recall any prior code you may have seen and you must write completely different code than anything in existence or face copyright litigation?
Nor are LLMs novel in any way when it comes to copyright violation. Microsoft is already rich enough they could ask developer(s) to reproduce a GPL code base but slightly different and then make an argument of coincidental, parallel thought. Or just pinky promise "we totally didn't steal this." Heck, it'd just be easier to copy and paste without license headers and then just run some basic human-readable obfuscation.
Furthermore, AI still isn't even close to being able to pull this off for even remotely complex projects. The context sizes are too small, the models aren't sophisticated enough, etc. They save time as an assistant, but novel output is extremely finicky and prone to errors. Getting a model that can be trusted to do what you're saying would be approaching AGI, in which case the economic ramifications are far more concerning than those on OSS.
Finally, how exactly would this "disassemble" OSS? The GPL has been violated for forever, especially behind closed doors. If this would kill it, the GPL is already dead and capitalism killed it. After all, if Microsoft - a company with a revenue stream larger than countries - decides to steal from virtually any OSS projext and the developers - a gaggle of volunteers with few liquid assets - try to take them to court, they're about to engage in a battle of attrition with legal fees. The legal system notoriously favors the rich, which is a far greater threat.
Yeah, I wonder what GPL-ed code is actually infringed upon. I vividly remember the time Oracle claimed Google infringed their copyrights because they copied a couple small functions, and the tech community just laughed saying the functions were too trivial. (Mind you, Java was licensed under the GPL.)
People harping that the GPL is somehow more affected by LLMs is kind of misguided. If there is copyright infringement, then almost all other licenses would be infringed too, since the vast majority of OSS licenses require keeping the copyright notice at the very least.
And the enmity towards Microsoft, whose "fault" is just having acquired shares of OpenAI, as opposed to almost every other Big Tech company who also released LLMs (and obviously injested a lot of github code), feels like the early 2000s again.
If anything it's the end of OpenAI
Can I get a citation for this? I have seen it produce code similar to GPL code for trivial things but never a direct reproduction.
This seems like a key line.
I can get my browser to output GPL code with a carefully crafted Google search query, and there might even be unlicensed use of GPL code in my browser itself. But as I understand it, as long as I don’t copy GPL code, I should be fine.
In this case they’ve disabled the GPL filter, and gotten GPL’d outputs. Fair enough. The question though: when the filter is on, can I trust it? Not sure!
Secondly, if you take this seriously you should already being doing some kind of static analysis to detect it because what if one of your employees copy/pastes some GPL code from somewhere?
It's practically a non issue that people are blowing out of proportion.
I don't think it is. If it is legal to input copyrighted stuff into an LLM and have the LLM magically remove the copyright, then the very concept of copyright is dead. LLMs are then a copyright-laundering machine.
Worse: if it is considered "fair use", then I don't think that I could even sell my copyrighted material with a contract that says "you are not allowed to feed it to your copyright-laundering machine". If it is fair use, anyone can do it, right?
1. The LLM might output GPL (or other restricted) code if you turn off the filters.
This one I see as a non issue.
2. The LLM benefited from GPL code and various copyrighted content in its training.
This is more complicated and for the courts to decide. My personal opinion is that as long as you as the end user are not publishing the output copyrighted content it shouldn't matter, and if you are then the owner of that copyright should pursue justice through the existing processes.
But yeah, ignoring the issues others have is a good way of living in a world with no issues at all.
Althought at that point, a well-intentioned human seeing the repo would similarly be fooled.
Currently, when does reproducing code become a violation of the GPL? If I see an interesting function in a project under the GPL, can I copy it? What if I rename the variables? Or what if I reimplement the function exactly? Approximately? Add perturbations Etc, etc. I get ship of Theseus vibes.
Can you give me a prompt that will do this? I'm a tad skeptical this is possible atm.
OpenAI recently fed everything on GitHub into a language model which will cheerfully reproduce GPL'd software without license or attribution.
I think that's the end of "free software" in the GPL sense. There is zero point to writing AGPL on a program if copying it into a language model and then back out into a text file without the license is a legitimate thing to do.
I can't imagine this being countered effectively by lawsuit. Microsoft has built a weapon that they can use to disassemble open source software, I think they'll take the legal side of that seriously enough that GPL will be a distant memory by the time the courts conclude in either direction.