Your intuition is correct, fitting for the data will lead to this arising. However, you misunderstand the interpretation. What is being fit is exactly an encoding of the dataset (a dual of it, like switching from edges of a cube to faces), so the "coincidence" is which specific images show up with extremely high similarity after similar prompting; likely meaning nothing related showed up in the testing, so it never got pushed around and settled nearby. For many images, that encoding was lossy and sufficiently moved that they are not matches to the training data, however for some they have ended up essentially encoded in these "lines" (as your analogy goes) and can therefore be reconstructed with not that much effort.
But it's not an isomorphism. I have a function that has a range that covers every single number in existence.
Does that mean that function is A copy of every single number in existence? No. Is that function an isomorphism of every number in existence? No mathematician would use the term "isomorphism" in such a way.
You could say that if F(3) = 4, then (4, F(3)) are two things that are isomorphic. But, again, no mathematician would call F(X) isomorphic to 4.
Technically in terms of law, what this would mean is that USING the model to generate a COPY would be illegal. The model itself does not hold a copy until function application is executed. Therefore existence of the model itself IS not illegal.
To say it's illegal is to say that a copy of the mona lisa exists in paint and paint brushes. F(X) is the paint and paint brush. F(3) is the painting.
I will say this, I don't think most people in the jury will be technical enough to get this distinction and the technical part isn't even really important in my opinion. The laws should be made based off of what's better for society/humanity not what's technically valid.
Still my comment here is on what is technically happening which SHOULD remain completely separate from the politics surrounding it.
And, not to belabour the point, but a function F that has an isomorphism to the real numbers means that F does indeed encode all numbers (and, when defined as a set, exactly contains a copy of each). That's literally the point of isomorphism. To demonstrate simply, the integers have an isomorphism to the natural numbers: you count them in an alternating pattern. That's an isomorphism between them (and it need not be relation preserving). So, with that, you can interpret all integers as natural numbers; we say they're the same up to their isomorphism. Isomorphisms also exist for the real numbers with other structures, via bijective maps, and so forth.
I'm not going to respond to the other stuff since it's nothing to do with what I said or responded to (even when it was a misinterpretation).
My point of contention is that it's not a copy.
But imagine this scenario. The model produces a picture that is identical to a picture in the real world, but that picture was NOT part of the training data. This is certainly possible.
It's fuzzy but there comes a point where the layers of abstraction between two entities that are isomorphic become big enough such that they don't violate copyright law.
If a model were to produce a pre-existing image not in its dataset, then we can strongly suspect that two behaviours have occurred (for the sake of establishing provenance).
First, it was just the ever so small, non-zero chance that it was produced by a random process that was not based on anything learnt, and within the finite resolution and some accuracy of colour, happens to match up to something already existing which, for example, may have been produced independently by a human artist - say, after the dataset was collected, but before the model was trained, such that it is emphatically a "coincidence" and nothing else, no collusion, no learning, no intelligence. Just chance.
Second, and I believe far more likely, this image has been encoded into the model by indirect learning and proxy. For example, say some notable and famous work of art is not part of your dataset (which is believable). However, say that there exist, say, art that references this famous work in some way (as artists sometimes do), which may be up to the point of parody (or some a gag), or may be some small aspect (colour scheme, notable style, outfits, lighting & composition, certain people, etc). Especially if it is what we would literally call "influential", is it not possible to reconstruct the famous work (that was not in the dataset) by indirectly learning about it from other pieces? Now, exact (or near-exact) matches are unlikely, for the most part it would probably tend more towards similarly repeating the references and parody of its dataset, but I feel this would strongly increase the likelihood of randomly producing it as in the first behaviour, and we're inflating the chances certain works are reconstructed entirely by "coincidence"; at least, as according to all observers that merely look at the binary presence of the original in the dataset.
Is this a copy? Is it not a copy? In all honesty, the notion of copy or not is insufficiently defined, because there's always some angle that can be part of it, or may not be, as most copyright law is handled on a case by case basis. Is it a copy because its provenance was an attempt to copy? Was it a copy by cosmic accident or just an independent work? Was it a copy by inferred reconstruction, thus not literal copying from the source, even if a verbatim reproduction is made? Is a vector version of a raster image a copy, and vice versa? What is "sufficiently" transformative to no longer be a copy? Does a copy of the Mona Lisa exist in the mind of a person that experiences it, while as a subject in a photograph it has grounds to not be? I'm trying to avoid anthropomorphising it and state whichever way what this is, I'd rather just illustrate how what we learn, what we experience, what we produce replicas of, and all of this is generally part of an unsolved problem/field: epistemology. Do we understand diffusion models fully? No. Therefore, weighing on one side or another should merely be for the purpose of a leading research question, or to exist on either side of a dialectic; such as in the courtroom. Perhaps we're still just too immature on this topic to really say anything about copying.
Really, the only thing we do know about copying, is that you should never get caught doing it. (For legal reasons, that was a joke.)
Could you say that paint and paintbrushes encode every single painting on the face of the earth into it?
I don't think this meshes with our intuition of the word. The model requires input for the encoding to be complete. If we don't think about it this way then practically anything on the face of the earth "memorizes" everything else. A pencil encodes every sketch that has ever been drawn and will be drawn.
This is a semantic issue. What do we mean by the word "memorize" or "encoding"? Eventually the gap between two isomorphisms becomes so big that the words memorize and encoding no longer apply. NN's are right at the border of this demarcation, but if we want to be consistent then the answer should be that NN's do not encode this information.
On the other hand, a neural network is a function, because that is what it is constructed as, and what it is intended to perform as, and indeed what it "does". It passes the duck test for a function, and then some. A function has a domain and codomain/image, two sets it draws its inputs from and then produces output accordingly, which for the purpose of mathematical definition, are an intrinsic part of the function.
Therein, it may contain and encode exactly every single painting not merely on the Earth but in all possible existence. The rub is that you still need to construct examples by providing input from that domain to have an output in the real world --- whatever your belief in the Platonic ideal, it is agreeable that the infinite or absurdly large finite sets of input and output that are described are not fully reachable/depicatable in the real world until physically performed, which requires the cause and effect of first providing input into a machine performing that function (let alone the literal encoding of a dataset into parameters as per my prior comments).
So, in a sense, with a model that is defined as such, we can indeed state that a paintbrush does encode every painting into it, albeit merely as possibilities that much be realised. Once picked up, it then requires cause and effect to move from one (the input motions) to the other (a resultant image), which has then indeed made one of those possibilities real. The issue with such a metaphor is that the implied model is begging the question, it circularly supposes that the paintbrush is responsible for this, such that it can serve as a function analogue for the metaphor, such that the paintbrush is responsible. Hence why I say you're mixing metaphors, because you've combined two incompatible metaphors (a neural network is a function, and a paintbrush is a "function" due to the cause-effect analogy to input-output) using shared language, that being the word function; which I believe I've shown has very different meanings here in both contexts.
The end result is that this "intuition" of the word/world is correct, but only partially, because it has not yet recognised how these two uses are divergent as per above. Indeed, such a model of a paintbrush would imply everything is also functions that ties together their input and output as possibilities which are retained, with any inversion providing a way to reconstruct inputs from outputs, such as figuring out how specific brushstrokes may have been performed. The critical thing is, again, to not mix the metaphors of the real world and the mathematical world, because that mathematical model can only describe possibilities for our real world. After all, it does not do us much good to state that a pencil contains every sketches as possibilities, because until we make good on it, it's essentially just a truism from the model that hardly makes it any easier to produce such a sketch - the same can be said for every drop of water, and so forth. Therefore, this is not a semantic issue, but a conflation of Platonic idealism with reality such that much is rendered meaningless, when reality has not provided incontravertible evidence for this idealism.
For example, the paintbrush might not have, as possibilities, every single painting. One approach would be to say it is only going to be used for one, and so a more appropriate model would be needed such that "no man stands in the same river twice" and so forth, and so this function metaphor would be demonstrably inappropriate. In contrast, the neural network (or a diffusion model) is still a function in a world that does this, because its parameters have been frozen and its random seed coupled to its input, with no recurrence from any prior usage as part of its operation. Therefore, its model as a function is exact and clear.
This means that "memorise" and "encode" can have very precise definitions for neural networks or diffusion models (as in the paper, memorised images being those reconstructable by given methods to within a known measure by some metric of accuracy), whereas with a paintbrush these do not have such precise definitions (indeed, beyond being reliant upon certain world models that may or may not be incompatible, they at the very least do not have only one obvious interpretation of meaning, unlike a neural network as a function).
As to the final points, it would be more appropriate invoke the notion of transformative derivation and ask whether neural networks or diffusion models are "de facto" justifiable as "always producing derivatives" due to size. I would say your statement can be reconsidered in light of this, the above, and my prior comments, given that they empirically do not always produce derivatives (we can reconstruct input data from prompts alone), and so the determination of whether there is sufficient transformation to be a derivitive work would have to be handled on a case by case basis for each image produced, as with all things - you would not say that an artist be given the carte blanche right to declare all their works wholly original or transformatively derivative when they've clearly just produced a copy of the Mona Lisa.
Additionally, if we were to "be consistent", it would not follow that this implies there is no encoding of such, only that they do not encode images they cannot produce. There is indeed exactly an encoding of possible (and memorised) images, because this is consistent with the mathematical model of a function that the neural network represents. We can state that neural networks do not directly encode all this information in their real world instance, as they do not, their parameters are only directly correlated to the dataset that is a subset of its input domain. However, they do still indirectly encode a much larger dataset than is explicitly given, due to the fact that humans do not produce art completely randomly, and like the paintbrush, we have created art (now used in a dataset) that is not simply a function but a real continuous object that has recurrence and relation to prior and simultaneous art and thought, and this is what gives rise to all their possible outputs that are not exact copies of their inputs (and by combination they should outnumber those memorised works as well). This is as per my previous comments, however to state explicitly in combination with the above, this implies that neural networks do exactly (if indirectly) encode every possible image they may produce (and not every image that may be possible) as the real world instance/approximation of the function it is depicting (which would encode every possible image, not merely those realisable, supposing its codomain/image was an uncomputable set), of which some are memorised by more direct encoding and others are inferred reconstruction. Either way, these images are producable by the model, therefore the model does encode it, either from the perspective of it as a function, or it as a realised machine performing that function with fitted parameters to enable it (within some bounds and margin of error). So, I would say that you may wish to reconsider the statement that a desire to be consistent implies this information is not encoded, when consistency actually implies the exact opposite - I believe that statement to likely be borne from the same conflation as above, so perhaps this paragraph is a clearly redundant summary by the time you finish reading it.
y = 2x + 3
(x,y) = (1,5)
Paint and paint brushes can be thought of as functions. You apply the right inputs (brush strokes) and you get an output that is a painting.
Therefore could you say that all paintings in the world are encoded into the paintbrush? No. You can't
When you input a 1, it does a *2 operation and a +1 operation and produces a 3. Nothing is memorized. 3 arises as a side effect of unrelated transformation operations.
"*2" and "+1" are vocal cords. They operate on any number of inputs just like how your vocal cord operates on any number of nerve signals and air blowing through it.
If my vocal coords are never used to record a copyrighted song then no violation of the law occured. If I never use the machine learning model to generate an existing picture then no law violation occured.
The curve can still intersect a unique point.
Logically the mathematics behind it says that it can happen. You present evidence which is refutable by new evidence. Logic if correct, cannot be refuted.
Second your way of measuring the probability is incorrect. IF we were to randomly pick an image out of ALL possible images of 32x32 1bit images THEN your probability would apply but clearly machine learning doesn't do this.
The actual calculations behind the probability is actually highly, highly dependent on the dataset.
To take the 2D line analogy further if we perform linear regression on the points (0,0), (1,1), (2,2) (5,5),(329,329),.... and a bunch of points with the same x,y coordinates the line would touch ALL training set data points 100% of the time as the equation of the line would be y = x. This isn't even called overfitting, y = x is actually the ONLY solution available. There's no way to prevent this if the data is just really clean.
There's no hypothesis. My conclusion is not scientific. My conclusion is derived from logic. It cannot be disproved.
>The set of natural images is going to be smaller than 5.56×10^-309, but it's still going to be astronomically large. You're simply not going to get an accidental hit on a 512x512 image, even with training
It might be low probability to get an exact pixel perfect match. But a similar match that is more or less indistinguishable (or even a different but obvious reproduction) to the original from the standpoint of the human eye is a significantly larger probability.
Does the interpreter have 99 Bottles memorized? After all, it is a function (with a 3-element domain).
https://en.wikipedia.org/wiki/Pierre_Menard,_Author_of_the_Q...
Much of that space consists of pairs of data that aren't relevant. So to understand it in terms of the analogy of the line in 2D space... The only thing that's relevant are points near the line.
For example if we use the line to represent housing costs(Y) over time(X). You can take a bunch of housing prices sold over time and plot it on the graph as dots. Linear regression will form a line that best fits those dots. Not EVERY single section on the plane matters though. Only the dots matter and the space that's very near the trend line have anything meaningful in terms of data.
"Copyright laundering" will be the phrase of the era. Throw Picasso in with a thousand reproductions, wash it with DeviantArt, get Picasso back out. Does it matter than it's algorithmically derived rather than a stroke-by-stroke reproduction? There's probably already ample case law around reproductions to deal with this.
y = f(x) = x + 2
3 = f(1) = 1 + 2
I think the technical part of this is sound. People will just take the face value explanation which is I see a 3! therefore a THREE was memorized. But technically a 3 WAS not memorized. This is categorically true.
You are Right. The persuasive part of this argument is not very good though as you can see by this thread.
Think of the analogy of linear regression. You have X,Y 2D space. X is the input (the English sentence), Y is the output (the image).
You use training data, which is a bunch of coordinates in this 2D space (each coordinate representing an <English, image> pair) to generate a best fit line (aka the machine learning model).
The properties of that line fit exactly what's going on here. That trend line will not touch most of the coordinates, but it likely will touch a few. When you take an X from the coordinate that touches the line and feed it into the equation of that line (aka model) you will get a Y (an image) that matches the coordinate Y exactly because the coordinate and the line intersect. Makes sense.
This is why an image appears to be memorized. But really it's not memorized. The equation of that line is the ONLY thing memorized here.
Why does it happen more when the image appears multiple times in the training data?
Well this is exactly what happens in least squares analysis. The line is generated from an equation that involves the averages of all the training data (aka coordinates) and if you have many samples of the same coordinate (aka image) you will skew the average and therefore the line towards touching that specific coordinate.
There is no complete technical fix for this issue. With enough training data, EVEN when you get rid of duplicates, the line will likely touch or be very close to a coordinate.
If you think about it, for a 2D coordinate system you can select a bunch of coordinates that generate a line that the never touches a single coordinate. But this defeats the purpose of machine learning. You're suppose to sample data you have no knowledge about and derive a model from it.
If you can already pre-select images that form a model that never touches a coordinate, it means you already have an understanding of the model and can likely just manually tune all the weights by hand to get what you want. As regular humans, we don't have the super-human ability to do this. We can only do it for the simplest 2D example and all the higher dimensional stuff can only be understood by analogy and random sampling.
One thing you can do is just add more data to the training set. That will move the trendline and could shift it away from something it touches. But at the same time it could move it towards touching a new coordinate.