Well, it probably didn't memorize any single of them. One thing that has changed dramatically from back when I studied AI (a whole 5ish years ago, sigh...), to my shock, is that the number of training epochs has dropped dramatically. Sometimes down to just one. If it's only ever looked at your image and updated its weights once, could that really be enough to copy it? Only if virtually everything in your image is also present in thousands of other images, which would suggest there was very little originality in your photo to plagiarize. (This is certainly the case with Obama's press photographers - political press photographers aren't hired for originality!)

This is a very interesting thought. If the image is very generic, IMO it should not be copyrightable, it should be public domain.

> political press photographers aren't hired for originality!

Yes they almost definitely are, what else would they be hired for? You wouldn't watch the same news everyday (even if it feels that way)

> the number of training epochs has dropped dramatically

So? Lets imagine a documented nurodiverse person (lets use a photographic memory) "snapsohts" a famous "works" and then sits in a room, after 5 years training, to produce "some work" are they free of copyright claims? Have you not seen the nurodiverse person who draws the entire New York sky line from 1 helicopter ride?

Your approach seems non-sensical, 1 epoch - 10,000 epochs, it doesn't matter - its still a copy / derivative work (which may exempt it, fair enough).

Personally I think all current AI work is just copyright on steroids, DALL-E outputs Shutterstock logos given the right input and Githubs co-poilt is just a lawsuit waiting to happen (implying anyone has the funds to actually sue Microsoft these days, the T&C's of Github must be incredibly broad).

> Yes they almost definitely are, what else would they be hired for?

Craftmansship. They have subtle, but probably very good, intuitions about lighting, facial expressions etc. and also a lot of concrete knowledge about these things. All of which an algorithm can pick up on by examining thousands of images once each.

Once or many matters because if you just examine each data point one, there can be no overfitting in the traditional sense. And that's what shocks me about the "one epoch is all you need" fruits.

> Once or many matters because if you just examine each data point one, there can be no overfitting in the traditional sense.

What makes you so sure? If my algorithm was literally just storing stuff in a hashtable to look up later, you'd get overfitting from a single exposure.

Well gradient descent doesn't do that. And the models, while big in terms of parameter data, are not nearly big enough to actually store all the training data.

Think of it in terms of updating beliefs about the target distribution. With backpropagation, you predict based on the input, and update your beliefs according to how wrong you were. So in a sense it's unsound to re-use data - your beliefs already incorporate them! And traditional overfitting is all that - it's when you use up all the information in your training data. This was many people's objection to neural nets (and I thought it was a good objection at the time, and thought myself that the future lay with more "sound" methods, which performed better on most metrics anyway at the time, rather than with dodgy biomimicry which wasn't really even similar to biological brains at all).

But yes, there are other types of overfitting if you want to get philosophical about it. It's just that the one I and everyone used to worry about, from training too much on your data, just isn't important anymore. And most of those clever principled and less-principled regularization methods just don't matter anymore!

I think the problem is with the definition of copyright. It doesn't apply to the current environment. It could even be the case, that it never made sense in the first place, and only now, we are realizing.

I have never seen Obama in real life. If I can draw him because I know him from copyrighted works, is my draw copyright infringement?

Are you copying one or more of those works? If so, yes. If you aren't, then no. How do you know? Well as an artist, after thousands of hours of copying, and thousands of hours of drawing from imagination, you know which activity you are doing. As an untrained person, you might not know, but the quality/value is so low no one cares.

But drawing from imagination is just copying from an abstraction existing in your brain. And that abstraction comes from inputs from the outside that you saw in the past, where would it come from if not that?

In the Obama example it comes from copyrighted inputs. Without the photos in the magazines and newspapers I would not have idea how Obama looks. Any draw I do, it's going to be derivative of those copyrighted photos and, maybe, other inputs more generics that I saw in the past (that could be copyrighted too).

How is that different from what that software is doing? It's not, it's just that it's done in an industrial scale.

This is just the same problem an artisan had when factories started to appear.

You are using the words 'input' 'output' and 'abstraction' interchangeably between the computer version and the brain version. We know next to nothing what those words mean regarding the brain. Declaring an equivalence seems like a substantial overreach. What we do do is use the terms from the latest technology to describe the brain[0]. What's particularly egregious about this iteration is we are also using human terms to describe the latest technology, and then using those terms 'inspired by', 'creative', 'intelligence', 'learning' as if the algorithms have anything to do with what humans are doing. And then even more, we go on to use them as smokescreen for piracy "the computer 'learned', therefore it's somehow not copying the source material".

Publishers license text to print in books. Music producers license samples to use in songs. Artists license photos and textures to use in illustrations and scenes. _License the source material used in training sets_. _Then_ go out and industrialize art, everyone wins!

[0] https://www.cl.cam.ac.uk/~jgd1000/metaphors.pdf

How does a human know what Obama looks like? They memorized thousands of images[1] of him, most of which were by professional photographers and, hence, copyrighted.

At what point is a neural network sophisticated enough that it can be compared to human learning for copyright purposes?

[1]: For most people at least. Some will have actually seen him in person, and for those this statement would of course not be true.