Re:Invasive Diffusion: one unwilling illustrator found her t...
by angusturner on Wednesday November 02, 2022 @07:35AM
(#33434121)
I mean, in a sense yes the training set is gospel. But these systems are also (generally) tested against held out data.
When you have to model 100TB of images with 4gb of weights there is no way this is possible without learning some kind of patterns and regularity that generalise outside the training set. Most generated items will be novel, and most training items will not be reproducible.
It doesn’t seem radical to suggest that the copying issue will continue to recede as we get better models.
And there are areas of research specifically concerned with _provably_ showing that you cannot identify what items a model was trained was.
Lots of reasons to be optimistic in my view.
As for actually using the model... personally I still find that unacceptable. Even if the risks are low, we have lots of works in the model, so the risk can still add up.
The idea you have in your head of training without regurgitation is likely not possible. The underlying technology treats the training set as gospel: the system is trained to regurgitate first, and generalization is a happy accident. Likewise, we can't look into a model to check and see what it's memorized, nor can we trace back an output to a particular training example. Which has ethical implications with the way that AI companies crawl the web to get training data; such models almost certainly hold someone's personal information and there's no way to get it out of there aside from curating your training set to begin with.