Alternatively, how many layers down do you have to go of NNs training other NNs before it's ethically okay? If your answer is "infinite" then that is the ethical equivalent of homeopathy where a solution of a bunch of random chemicals is diluted to the point that there is 0% of the original left in the sold product. Interesting question.
It's all fruit of a poisoned tree.
It's an difficult ethical dilemma. How are we categorizing the harm done to victims of child pornography? Is it based on how many different people see the same image or just the number of times the images was viewed in general? Obviously the victim isn't getting some sort of electric shock every time somebody sees the image.
Do we include it additional harm if the identifying features of the child in question are unavailable? What if the victim doesn't know they were part of the training model or that there are even images of them on the Internet? Would it be ethical if training data from children in non-pornographic situations was used? What if somebody had child pornography made of them but allow their image to be used (donated?) after they were 18?
I'm not claiming to know the answers.
The incident with YouTube was a sign that some people are a little disconnected from society. They went around random videos uploaded by children posting flirtatious comments and highlighting timestamps of them in compromising positions.
You could argue them viewing the videos alone was immoral. But their conduct was just atrocious. I don't know how it got into their heads that something like that is acceptable. I really hope we could teach them a thing or two about common sense, regardless of whether viewing images is moral or immoral.
Training models is complicated. Whoever it is is going to want to have the rights to their likeness. This would be after they are of age. In theory, you may be able to teach an AI with many images how to produce a generic person.
They will never find out is a bad mindset. It is true in many cases they won't find out and they may be better off not knowing what someone is doing. But when and if they find out, it will happen in the worst possible way at the worst possible time and it will blow up really badly.
In the extreme, to produce better models, your business would eventually be incentivized to create new data... but if you limit yourself to the usage of data collected as a byproduct of stopping criminal activities, it's not clear to me what the morality case should be.
A historical example might be the nazi human experiments -- the experiments themselves were horrendous, but now that they've been done and we have the results, is it unethical to use that as the basis for further, ethical, research? That is, would it be unethical for me to read the outcomes?
It's a very complex issue. My read of the linked article is that, at a minimum, the victims whose likeness is used should consent to its use. Even given that, I'm not sure that our society would tolerate such an endeavor.
Assuming this is a course people would agree to take, would it faster to convince society to adopt a new mindset or would it be faster to create ethical alternatives?