> may have been used

That statement is carrying a lot of non-evidence with it.

Say my fetish is chocolate. If I ask an image generating AI to give me a photo of an adult made of chocolate and covered in sprinkles, that doesn't necessarily mean it was trained on that specific thing.

Thank you for dodging my point with a non-sequitur. The contrived example does not diminish the potential real harm that exists in the original use case.

> potential

You don't seem to understand that "potential" is not "actual" (and also conveniently says nothing about actual probabilities). I could claim almost anything is "potential".

Eating grapes could "potentially" result in choking. Living in a house made of wood could "potentially" result in dying in a house fire. Driving over 70MPH could "potentially" result in hydroplaning if it is raining.

Why do we not care more about the "actual" harm (and its probability), as measured by science?

Oh, of course- Things that are banned are difficult to study, which is why people can invent any beliefs about them that they want to.