Not true at all. A camcorder creates an exact copy of the movie. If that's what training models was doing, I would agree it's copyright infringement.
A camcorder bootleg of a movie is objectively not an exact copy, and subjectively a worse reproduction than GPT's from-memory Mona Lisa. But it's close enough to count for copyright violation purposes.
I'm not claiming the weights of a model can exactly reproduce every work they were trained on, that would of course be an impossible violation of information theory.
But clearly they are capable of making extremely close, close enough to count, reproductions of some works they were trained on. Is it not fair to say the weights "contain" a lossily compressed copy of the Mona Lisa since I can spit that image out of them? And if they contain the public domain Mona Lisa, do they not also contain countless copyrighted works, which a thin band-aid of system prompts tries to prevent them from offering up on-demand?
I'll admit, this technology is not exactly like copying stuff, but it's not quite not like copying stuff either. It is a new gray area. What I totally reject is the claim that it's just like a human learning things and should be treated the same. Unless the trained model was never allowed to be copied to another machine, and had to live and die exactly once with the hardware it was trained on. And had rights of its own. All of which of course would be absurd.
> It does scale, but I don't think that's particularly relevant. It's not any reason to make it illegal. If something that's fine and legal becomes effective enough to threaten other people's business models, that should not make it illegal. That should just make it the dominant business model and the other business models should lose.
Absolutely it matters! You can't spell copyright without "copy". Before the printing press there was no reason for such a thing. There are only extremely rare cases of disputes over manually copied books and certainly no system of law to protect it. It is the scaling factor of mechanical copying becoming possible that made the concept of copyright necessary in the first place. Scale is everything.
This would indeed be absurd, but these are arbitrary differences. One can always point out differences between two different things, but that doesn't make those differences relevant to the issue at hand. Copyright law has little to do with whether someone or something has rights of its own, or lives and dies in a single body, etc.
The only differences that matter here are whether or not an actual copy is being constructed and/or used in such a way that it infringes on copyright law. (And, arguably, whether this copyright law is just or good in the first place.)
I don't think it is or should be.
It is possible for LLMs to reproduce copyrighted works sometimes very accurately. And you could maybe argue that this means they possess some copy in their weights. But a human brain also possesses the capability to memorize an entire book and then reproduce that book's content, and I don't think that's grounds for any sort of issue. In fact, even a regular computer without an LLM can do such a thing, and that's fine. In fact, regular computers have even been used to do such things, without it being copyright infringement. (See: Google Books.)
What differentiates the human brain and an LLM from a camcorder is that a camcorder exists for no other reason than to copy. An LLM and a human brain can do much more. And training a brain or an LLM is done not in order to produce copies, but in order to do something far different, far more expansive, and far more impressive.
The fact that "well maybe technically there is something resembling a copy in the weights" is not really grounds for copyright infringement. Nor is, "Well even though LLMs can do a million impressive things, they can also sometimes make an exact copy."
If somebody did a training run of some new kind of model that literally existed to do nothing but reproduce what it ingested, then sure, you'd have a point. Hell, if someone uses a normal LLM to reproduce copyrighted works, you have a point in those uses as well.
But 99.9% of the use of an LLM is to have it solve novel problems and create novel things, based on what it learns. And that's 99.99999% of the reason for the training, too. Some very incidental I think that pretty clearly holds up as not infringement.
It's a bit like pencils. Can pencils be used as weapons? Sure. But we shouldn't be banning them from schools.
> Absolutely it matters! You can't spell copyright without "copy".
Not to nitpick here, but as far as I'm aware, under U.S. copyright law at least, there is no general rule that copying becomes infringement only after some threshold number of copies. People get arrested or sued all the time for making small numbers of copies. You are right that, perhaps the impetus for copyright law in the first place, came as a result of mass copying. But it is not the scale that supposedly makes it a moral hazard, or even a legal issue.
I did read your whole post, but I only want to address this last point because I think it gets to the core of our disagreement (or maybe a little bit of agreement?)
Before the printing press there was no need for copyright because copying itself was extremely high effort and non scalable. If somebody manually copied a manuscript, fine, good for them. The printing press changed that. Now somebody could free-ride with and financially benefit 1000x from somebody else's work. This seemed to be unfair and bad for society. Copyright followed.
This is what learning a skill was like. Sure, somebody else could "copy" your learned skill and build the same skills. But it's so high effort and non scalable for each new person to be trained, that was never a problem. Good for them. The LLMs change that. Now the companies selling LLM intelligence are free-riding and financially benefiting 1000x from many other peoples' work. This seems to be unfair and bad for society. <Something> will follow.
I'm not sure what the <something> will be, that is beyond the scope of this conversation. I don't think copyright law, designed for another era, has the answers to this. But I do think there should be and will be a <something>.
I've been studying business in general and startups for probably 20 years now. And if I've learned anything, it's that the popular conception of what it takes to create something and succeed in the capitalistic marketplace is basically just to create it.
The popular conception is that the creators are the kings/queens.
But reality is far different. You not only have to make something, but you have to make the right thing that the market actually needs and wants. And it can't just be the right thing, but it also has to be pretty close to the best thing. And it can't just be the right thing and the best thing, but it also has to be fairly unique and differentiable. And making something that's differentiated, unique and high quality usually isn't good enough either. You also have to work out distribution, marketing, sales, advertising. You have to figure out the right business model. You might need funding. You need maintenance, customer support, a million other things.
Business is hard.
And in my opinion, you don't really "deserve" anything unless you do all of it. And the market itself is a good enough barometer of what you deserve. If you do all of it, the market will reward you. If you're not being rewarded by the market, there's almost certainly some shortcoming that you have. In rare cases, someone is playing unfairly and that's why we have laws, which by and large work for the majority of businesses to protect them from fuckery, sabotage, etc.
But not, imo, from superior competition.
There are some industries that have figured this out. The vast majority of software engineers, for example, do not expect to be rich simply because they created a popular program or library. They understand that they need to do the marketing, the sales, the fundraising, and all of that. Even though they're programmers and abhor this kind of work, and would much prefer to just do the coding.
There are some industries where the culture has not figured this out as much. Music is a good example. There's still a common belief among musicians that having a good voice and making good music should be enough. Even though it's an incredibly cutthroat competitive market and doing the actual marketing and distribution to get a song out to the masses is 98% of the work if you want financial success, because there is a huge supply of good singers. The musicians who make it typically team up with the "suits" who are good at the marketing and distribution. Or figure out how to do it themselves.
Business success is hard, by nature of it being competitive.
I don't see any problem with this, and I don't think we should be creating laws that restrict the rights of the general population in order to make things easier for a select few who are trying to profit.
See my comment here: https://news.ycombinator.com/item?id=50567495
I think if you want to build a better business, you should have to build a better business. You shouldn't be able to just create something, and sit on it, and make it illegal for the rest of society to compete.
And I think it's kind of a cancer that we allow that. Honestly, I think copyright law is the biggest thing holding artists back. I really don't think it's coincidence that these industries that have the most entitled (imo) creators who think they deserve to get paid for simply creating stuff, rather than for building a successful business around it, are the ones that have a strong culture of relying on "IP".
If we just got rid of it all, I think we would see a new generation of talented and empowered artists who are also entrepreneurs. Just like we do in software and restaurants.
> Should these people be illegal? Is it suddenly copyright infringement for them to behold copyrighted work? Why or why not?
> If you say, "Well they're people, not machines," that's true, but why should that lead to a distinction? What's wrong with machines doing things? Should we make calculators illegal? Obviously not. So what's so wrong with machines that can think and learn and do so effectively?
Responding separately to this part that was added. That's a great thought experiment. I liked thinking about it.
I would NOT say these people should be illegal. I would NOT say it should be illegal for them to behold copyrighted works.
I would say they WOULD own the rights to anything they create that is a novel, transformative work, and that they WOULD NOT own the rights or be legally able to profit from anything they create that is clearly a 1:1 copy of other people's work.
It would be immoral and illegal for me to run a business wherein I force this person to view millions of copyrighted works, then enslave them to create new works for my customers on demand.
Basically, the same rules I apply to any other person today.
Do you apply all of these rules to machines, too, or only some of them? I don't apply all of these rules to machines. Like you said, my reason is just "Well they're people, not machines". I accept that. What about you?
Like I said in my other response, there's always going to be arbitrary differences between people and technology and if you pick some random difference you could try to make an issue with that, but that difference is not what's at hand. It's not about the fact that technology is a choice and people don't. (Which is where I think you'r getting this slavery thing from?) Of course it would be unethical and immoral and should be illegal for you to enslave someone. But that's not the question we're trying to answer today.
What we're talking about today is the copyright infringement of it all. So let's take slavery out of it.
Let's say this person simply worked at a company where they viewed millions of copyrighted works and created new works for the company's customers on demand.
What would be unethical or illegal or immoral about that?
Even if they occasionally reproduced an entire copyrighted work, I could grant that that would be an issue, and shouldn't be allowed under current law, and action could be taken. But I couldn't agree that their mere ability to do this, or the fact that they do this occasionally, should mean it was illegal for them to do the learning and training in the first place.
> Basically, the same rules I apply to any other person today. Do you apply all of these rules to machines, too, or only some of them? I don't apply all of these rules to machines. Like you said, my reason is just "Well they're people, not machines". I accept that. What about you?
I think it depends on the question at hand, not on fundamental differences between man and machine. Of course man has choice and freedom and machines don't, and that's fine. Therefore I don't think I wouldn't compare using a machine to slavery just because you don't have to pay a machine and have to pay a man.
Any analogy has some differences that simply don't translate or hold up. To me I'd just like to focus on the relevant parts of the analogy.
When it comes to laws, I don't see the point in making laws that apply to machines but don't apply to humans doing the same thing.
For example, I can understand not allowing self-driving cars by default, because humans require licenses and training and are subject to jail time for misbehavior, whereas self-driving cards are fundamentally unable to go through the same processes and be held accountable, so society needs time to adjust.
I can understanding allowing human telemarketers but not allowing robocall telemarketers, because the experience of talking to a human is fundamentally difference than the experience of talking to a robot, and it's a human on the other line who has to deal with the robot.
But I couldn't understand, say, allowing humans to calculate numbers, but not allowing spreadsheets or calculators to do it. Sure, they're advantageous. Sure, they put people out of business. (Famously, "calculator" used to be the name of a job, not a device. RIP.) But it's not directly making anyone's lives worse, outside of being outcompeted in the market, which imo is fair and overall beneficial to society.
But I can't understand allowing humans to learn so they can acquire general intelligence, but (imo) abusing copyright law to say that robots aren't allowed to learn, because they're too good at it and (once again) outcompete some jobs or roles in the market.
When we talk about a person learning/recreating stuff they learned from, the person themselves is who we are holding responsible. But a machine is not guilty of anything. It is owned and wielded by a person or more likely a company. It is what that company does that may be unethical.
This goes right to your person-employed-by-company example, I do think the company is acting immorally by gathering up and feeding all that copyrighted data to their superhuman employee, knowing perfectly well that what they are selling their customers is the employee's mimicry of it. It wouldn't be illegal, but then, we don't have the precedent for it since the superhuman-mimicry biotech doesn't exist.
> And this is pretty much exactly what training a model is.
Equating model training to a human learning a skill is like equating taking a camcorder into a movie theater, to a human remembering the movie they watched and imagining scenes from it in their head.
It scales. It has very different consequences since one trained model can be copied infinitely and obsolete an uncountable quantity of equivalently-skilled humans. It also is capable of imitating on a much finer level.
Here[1] is an image of the Mona Lisa I requested GPT6.1 Sol Medium to create as accurately as possible without referencing any external sources, using only its training. It claims to have followed my request: "Generated from training alone, without web searches or reference images."
Of course, the Mona Lisa is public domain, which is why I didn't have to fight it to do this. And I'm sure it is represented in the training data thousands or millions of times, unlike, say, my obscure open source libraries. But the point of the exercise is to show the "trained" weights can reproduce what they were trained on in a way far superior to any human memory (could any artist recreate Mona from memory this well?) and arguably, even better than some technologies we do consider copyright violations such as a very badly shrunk-and-compressed JPEG or a picture-of-a-picture-of-a-picture.
[1]https://imgur.com/a/QaGHEMi