> If you don't own the copyright to your entire training data set, you are infringing by publishing something the model produces as I see it.

I suspect that would be an absolutely terrible standard in practice. I realize you specifically have deepfakes in mind, but consider that in more general terms those are outputs from a machine learning algorithm. Such outputs will inevitably fall on a continuum regarding the degree to which they resemble a distinctive art style, voice, personality, or other metric.

It seems unreasonable to me that it should be illegal to use arbitrary input data to train a NLP translation model or an image classifier. Farther along the spectrum of outputs, style transfer GANs don't seem like they should be a violation of the law. Should TWDNE really be against the law to host? (https://www.thiswaifudoesnotexist.net/) According to your proposed standard, it would be.