> If you don't own the copyright to your entire training data set, you are infringing by publishing something the model produces as I see it.

Imagine if that were true - a human artist couldn't create anything, because everything in the world around them is their training set!

I think it has to be a likeness test - if an average person would think its a real Jay-Z music video, then it's a derivative work. Don't muddy it with the implementation details of how it was made.