Like if you set it to a threshold where it's not routinely catching false positives, it doesn't catch the original images either because they're still too subtly different or something.

Maybe they really did just not bother though. Heck, maybe they want to test it in court or something.

But not at a corporate level. There's no universe where the company hasn't gotten together lawyers to analyze the risks and investigate whether they should implement technical safeguards to help mitigate them.

In this situation it's 100% a choice, not somebody just not thinking of something.

Google can legally return a copyrighted image in Image search (it’s not selling the image, it’s selling ads against search results) and this would probably fall under the same protection.

Now if stable diffusion was sold as SaaS in the way Dall-E is…

First, it probably wouldn't work. Stable Diffusion is going to be a lot better at telling whether two images are the same than a perceptual hash check. E.g. I bet if you removed all pictures of the Mona Lisa from the training data, it could still produce a pixel perfect copy of it, just from the many times it appeared in the background of pictures, or under weird adjustments or lighting that fooled the perceptual hash.

Second, I would guess they wanted common images to appear in the training data more than once. It should be trained on the Mona Lisa more than someone's snapchat of their dinner. Common images are more salient.

Reminds me of asking Chat-GPT for an obfuscated C contest entry. It produced one verbatim, though it couldn't explain what it did (it looked like syntactic nonsense, but it printed the Twelve Days of Christmas when run). I can only imagine it saw that inscrutable sequence of characters enough times that it memorized it.

Why would it be? Stable diffusion is a text-to-image model it's not at all focussed on determining whether images are the same.

Secondly, I'm not proposing deduplication of the training set (I know some sibling comments have proposed this). I'm proposing a perceptual hash or similar check on the way out so that if a "generated image" is too similar to an image in the training set it gets dropped rather than returned.

And it could go the other way too; it could be that the perceptual hashes are even "better" than humans at seeing past those distortions.

My point is that this all gets more complicated than you may think when you're trying to apply a hash designed for real images against the output of an algorithm.

And even if the hash worked perfectly, the false positive and false negative landscape would still be very likely to contain very surprising things.