> Stability AI is happy to follow copyright laws for their music model, because they know that music labels will hold them accountable. So this seems like a good time to point out to larger companies like @WaltDisneyCo that their copyrighted material is being stolen and used too

I mean this is a pretty good point. If they're so sure this is legal, then train on copyrighted audio+video media as they already do with copyrighted visual media.

Avoiding doing something because you don't want to get sued and subjected to a lengthy court battle is completely rational and it doesn't mean that doing that thing is illegal.

For example, for decades many TV shows came up with their own lyrics for Happy Birthday song, even though it was well known that the song wasn't copyrighted, because nobody wanted to get sued and fight that battle. Easier to just change a few words in the script.

It's not just legal issues that'll lead to self-regulation. It's also stuff like this:

https://imgur.com/a/vAqCUP7

Simply put, there is less information in music. Copyright infringement cases have been won on as little as a handful of notes. So the likelihood of a copyrightable element from the underlying showing up in the model's output is that much higher. A single melody represents a similar portion of all possible melodies as does a color all possible colors and certainly less than the fraction represented by a single color palette. Yet you cannot copyright a color palette but you can a melody.

Copyright law regarding audio media is entire different beast than visual media; the case law and fair use technicalities do not always line up as simply as "if its legal on visual media it must also be legal on audio."

This is apples and oranges. SD et al are defended on the grounds of being transformative use (https://en.wikipedia.org/wiki/Transformative_use): they do not distribute (ie copy) the original training images, and they are not a derivative work due to transformativeness, so the license of the original images is completely irrelevant. (Details like 'signatures' are also irrelevant: if I write a style parody of William Shakespeare and add a '--Willy Shakespeare' at the end to round it off, have I revealed that I have secretly copied his work? Of course not. It's just plausible that there would be a name there, so I came up with a name.)

The criticism here is that distributing (copying) the original image violates the non-commercial clause of the original images because someone, somewhere, might somehow have made money in some way because the dataset exists; but as they somewhat lamely acknowledge later, what counts as 'commercial' has never been clearly defined, and it probably can't be defined (because for most people 'commercial' seems to be defined by 'ewww'), and this is why CC-NC licenses are heavily discouraged by WMF and other FLOSS groups and weren't part of FLOSS from the beginning even though Stallman was in large part attacking commercial exploitation.

Does anyone know if attempts have been made to trick these ML models into reproducing original copyrighted inputs verbatim?

That's a known failure mode called "overfitting" or "memorization", where a specific input text is very accurately reproduced.

I'm not aware of it occurring for any copyrighted inputs, but it occurs for many famous artworks-- it's nearly impossible to convince Stable Diffusion to style "Mona Lisa" at all.

That's not really how memorization in neutral networks works. For classifiers, memorization is more like learning a hash function and a lookup table; no need to store the full image at all. Even for very large models, the weights are a tiny fraction of the size of the original data.

It's probably helpful to think of embeddings for generative models in a similar way; it's a very specific embedding function, like a locality sensitive hash, which doesn't require actually storing the data.

Thanks. Yes, I shouldn't have said "verbatim" -- I meant something more like "close enough to cause legal trouble". Obviously copying verbatim is a violation of copyright, but there must be some threshold of "close enough" that is still problematic. E.g. compressed MP3s of copyrighted songs aren't a verbatim reproduction, but as far as I'm aware they're still covered by copyright.

Trademarks are even broader.

A band playing publicly a reinterpretation of a famous band will get sued even if its not verbatim.

That’s an even better example.

One, the diffusion model's possible output space contains every RGB image ever. But two, it cannot ever possibly contain the original inputs verbatim, because (the size of the model)/(the size of the training set) comes out to be something like 0.2 KB per image. Unless it's an incredible compression algorithm, diffusion necessarily have learned something from the input rather than copy-pasting things, as claimed upthread.

I edited my post a while ago, but I shouldn't have said "verbatim". See reply to sibling for more what I had in mind.

> Unless it's an incredible compression algorithm, diffusion necessarily have learned something from the input rather than copy-pasting things, as claimed upthread.

Arguably, "learning" and "compression" are the same thing.

In this sense, you can view SD as a compression algorithm where the decoder is the model, the compressed file is the prompt + tweakable params, and there aren't any error checks made, so you can feed random data into the decoder and get something out.

Or maybe the compressed file is the model, and prompt + tweakable params is a "path" inside that compressed file?

There's been a lot of work on memorization, yes, and you can also do nearest-neighbor lookups in the original data to gutcheck 'memorization'. As usual, the answer is "it's complicated" but for most practical purposes, the answer is 'no': you will get the Mona Lisa if you ask for it, absolutely, but the odds of a randomly generated image being a doppelganger is near-zero. (If you've seen stuff on social media to the contrary, then you may have been misled by various people peddling img2img or 'variation' functions, or prompting for it, or other ways of lying/ignorance.)

But you certainly can get things like watermarks without any real memorization. Watermarks have been a nuisance in GANS, for example - the StyleGAN CATS model was filled with watermarks or attempted meme text captions, even though the cats were so nightmarish that they obviously weren't 'plagiarized' so nobody made a big deal about it back then and they understood the GAN had simply learned that watermarks were a thing in many real images and it would try to imitate them where plausible in a sample.

If there's a specific prompt that makes the model produce such an image (without img2img), why shouldn't it count? The question isn't whether the model produces such things randomly, but rather whether it's capable of producing them in principle, even if it requires a very elaborate prompt.

> If there's a specific prompt that makes the model produce such an image (without img2img), why shouldn't it count?

There's a specific prompt which makes a human artist produce such an image without img2img too: "Please draw the Mona Lisa". There - you just did it in your head while reading this comment!

So? Humans aren't copyrightable works. But a bunch of weights that constitutes the model is just data, and that data can very well contain copyrighted works. The question is whether it does in any meaningful sense. And if it can reproduce them verbatim with the right prompt, I don't see how the answer could be "no", anymore so than a password-protected archive of a copyrighted JPEG.

>if I write a style parody of William Shakespeare and add a '--Willy Shakespeare' at the end to round it off, have I revealed that I have secretly copied his work? //

I doubt you're suggesting SD, Dall-E, etc., are producing parodies so bringing in parody considerations muddies the water a lot. Also, Shakespeare's works are out of copyright.

If you sell a painting signed with a [facsimile] signature of Dali then it's pretty hard to say you didn't copy the signature, as a minimum. Thats likely to be a trademark violation too. Now, suppose you include aspects in the image specifically associated with the artist, and a signature, ... there's no way to genuinely deny that is a derivative.

> a painting signed with a [facsimile] signature of Dali

That's not what's happening here though.

If you look at the original tweet (https://twitter.com/LaurynIpsum/status/1599953586699767808) it seems that the complaint is about the "mangled remains of an artist’s signature". I don't see any examples where it's actually copying the signature of a specific artist.

(Please do share an example of that if there is one.)

I do respect artist's concerns. I have a hard time getting this one though. The ai learned that humans usually put squiggly lines in the corners, and it does, too. What is wrong with this?

It's reasoning based on a false assumption. If you start from the assumption that AI imagery is based on specific source images, which had signatures, then it can make sense to assume that the squiggly lines in the corner are evidence that it copied-and-mangled some such specific pictures.

That's not what it did, of course. But if you start from the assumption...

> distributing the original image violates the non-commercial clause of the original images because someone, somewhere, might somehow have made money

I agree with the rest of your post, but this point seems a bit uncharitable.

I think the claims would be:

1. It's a breach of copyright for Megaface to share the images in any case without attribution & replicating the CC-NC license. It would (presumably) be OK assuming Megaface were to correctly apply the CC-NC licenses to the dataset.

2. It's a breach of copyright for anyone consuming Megaface (e.g. Google) to use those images for commercial purposes.

And your argument for SD applies to 2. that regardless of license, it's OK to create a transformative work. But it still doesn't get Megaface off the hook for 1. - distributing those images without the license.

Nobody is complaining about William Shakespeare or public domain works that are out of copyright. The issue is that there is clearly copyrighted works in the model that living artists have not consented to being in the model.

> SD et al are defended on the grounds of being transformative use they do not distribute (ie copy) the original training images, and they are not a derivative work due to transformativeness, so the license of the original images is completely irrelevant.

If this is irrelevant, why is Stability.AI creating an "opt out" system for artists? It's almost like they know they are copying digital artists works without their explicit consent. This wouldn't be an issue if they only used images in the public domain.

https://www.technologyreview.com/2022/12/16/1065247/artists-...

And we both know that Stable Diffusion have Dreambooth which you have to pay to use the platform, so this is clearly not fair use.

https://platform.stability.ai/docs/getting-started/credits-a...

> If this is irrelevant, why is Stability.AI creating an "opt out" system for artists? It's almost like they know they are copying digital artists works without their explicit consent. This wouldn't be an issue if they only used images in the public domain.

It might also be because there's a huge backlash against the concept of stable diffusion in certain circles. Whether or not that backlash matches the legalities, or if their logic is sound at all, has absolutely nothing to do with it.

> Nobody is complaining about William Shakespeare or public domain works that are out of copyright.

They definitely are, and I deliberately chose a public domain writer for my example to get away from the narrow procedural grounds and focus on the issue: does generating a 'signature' or 'watermark' prove copying? No, of course not, and that's true whether you are in or out of the public domain.

> If this is irrelevant, why is Stability.AI creating an "opt out" system for artists?

What is prudent or nice or moral or good PR != legal. (And vice-versa, of course.)

> Even with a fair use argument, there are examples in cases where AI was generating art that included the signatures of artists.

I went through the post, and I am not sure whether I agree with the analysis of the examples. Diffusion models are conceptual parrots, and it is possible that "25% images contain a scribble in the bottom right corner, so the model will make a scribble in the corner" is what is being construed as a signature in this post.

I think a large part of outrage from the artists about diffusion model "stealing" art comes from a place of disbelief that machines can be this good without "stealing", and it's perfectly natural. In fact, it's unnatural to me how good machines have gotten in image generation, and it is a field I've been following for five years now. However, because I understand the model and can implement it myself, I can convince myself it doesn't need to steal, just needs to be able to model correlations at some ungodly level.

>I think a large part of outrage from the artists about diffusion model "stealing" art comes from a place of disbelief that machines can be this good without "stealing"

I think when you make machines that automate away some people's passion and purpose in life, of course they're going to be upset. When, on top of that, the machine automating their work is a "conceptual parrot" that is parroting the concepts they invented without their permission, of course they're going to be pissed off.

Besides, whilst AI image generators don't steal exact elements from the training data, they do basically steal the artstyles and subject matter of illustrators, which takes them many years to foster. Imagine an illustrator of fantasy book covers hardly being able to find work anymore because some publishers figured out that, instead of hiring him, they could cheaply hire an unskilled person from a third world country to type the illustrator's name into an AI image generator so that it imitates his works, along with a few keywords for the book in question, until eventually something good pops out. That is currently considered fair use in US copyright law, but in my opinion it is nonetheless so unjust that it justifies calling it "stealing".