The NCMEC hash list is private, and adversarial attacks require running gradient descent and being able to generate a hash value for arbitrary input.

At least one of these two things must be true: either Apple is going to upload hashes of every image on your device to someone else's server, or the database of hashes will be available somehow to your device.

Turns out there's a third option I wasn't thinking about: private set intersection.

https://news.ycombinator.com/item?id=28097683

Is it possible to narrow in on a hash using gradient descent? You can correlate distance between inputs to distance between hashes somehow?

Replying to my own question since I can’t edit anymore: it turns out “perceptual hashing,” which I didn’t know much about, has exactly this property, that small changes in the input result in small changes in the output.

Might be hard if they use a huge hash.

One thing to note is these are not typical cryptographic hashes because they have to be able to find recompressed/cropped/edited versions as well. Perhaps a hash is not an accurate way to describe it.

There have been a number of cases where people have found ways to trick CV programs in to seeing something that no human would ever see. If you were sufficiently malicious I imagine it would be possible to do with this system as well.