News for hackers. Stuff that matters.
Re:Apple’s new abuse prevention system: an antritust/competi... by DSingularity on Friday August 06, 2021 @08:37PM (#28094486)
No. You can design a system where the FPR is essentially zero. Even if shitty md5 is used.
Re:Apple’s new abuse prevention system: an antritust/competi... by Conlectus on Friday August 06, 2021 @08:51PM (#28094587)
Perceptual hashes are not exact hashes, otherwise they would be useless for this task; you would just mirror the image or change 1 pixel.

They are instead fuzzy classifiers, and thus have non-zero error rates.

Re:Apple’s new abuse prevention system: an antritust/competi... by nomel on Friday August 06, 2021 @08:59PM (#28094656)
But this trigger requires many of these false flags to exceed the threshold (most likely why it exists). I imagine the "one in a trillion" numbers they claim for false flagging rate are probably cemented in reality, and make it trivial for human review.
Re:Apple’s new abuse prevention system: an antritust/competi... by skinkestek on Saturday August 07, 2021 @12:48PM (#28099957)
I've explained the problem here: https://news.ycombinator.com/item?id=28099927
Re:Apple’s new abuse prevention system: an antritust/competi... by DSingularity on Friday August 06, 2021 @11:26PM (#28095420)
How does this relate to the allegation I was responding to? There is a difference between false positives and false negatives you know. False negative is where the criminal evades the system through measures you describe. False positive — what I was responding to — is where Apple incorrectly accuses an innocent person and thus violates their privacy during the subsequent review.
Re:Apple’s new abuse prevention system: an antritust/competi... by Zababa on Friday August 06, 2021 @09:07PM (#28094701)
> No. You can design a system where the FPR is essentially zero. Even if shitty md5 is used.

How can you do that, considering md5 can have collisions?

Re:Apple’s new abuse prevention system: an antritust/competi... by schoen on Friday August 06, 2021 @10:47PM (#28095244)
A confusing thing that I think people haven't addressed clearly for the most part:

For a hash (whether cryptographic or perceptual), there is a chance of random collisions and also a difficulty factor for adversarially-created intentional collisions. The random collision probability has to be estimated based on some model of the input and output space (with cryptographic hash functions, you would usually model them as pseudorandom functions and assume that the collision probability is the same one created by the birthday paradox calculation).

Intentional collisions depend on insight about the structure of the hash function, and there are also different kinds of difficulty levels depending on the nature of the attack (preimage resistance, second-preimage resistance, and collision resistance). Gaining more insight about the structure of the hash function can act to reduce the work factor required for mounting these attacks. That should be true for perceptual hashes just as much as cryptographic hashes, but presumably all of the intentional attacks should start off easier because the perceptual hashes' threat models are weaker and there's much less mathematical research on how to achieve them.

And in AI systems involving classifiers, it was generally easy for people to create adversarial examples given access to the model. Perceptual hashes for estimating similarity to specific known images aren't the exact same thing because it's less like "how much like a cat is this image?" and more like "how much like NCMEC corpus image 77 is this image?", but maybe some of the same techniques would still work. In the cryptographic hash analogy, I guess that would be like trying to break preimage resistance.

Re:Apple’s new abuse prevention system: an antritust/competi... by Zababa on Friday August 06, 2021 @11:22PM (#28095398)
Thanks for the detailed explanation. I understand why that works for perceptual hashes if you make them really precise, however I doubt it would work with md5, which is why I asked.
Re:Apple’s new abuse prevention system: an antritust/competi... by DSingularity on Friday August 06, 2021 @11:35PM (#28095459)
The discussion I thought we were having was about false positives and not adversarially induced false positives. For the former the random collisions have a probability of 1/(2^64).

To mitigate adversarial false positives one idea is to use the combination of a cryptographically strong hash along with a randomly selected perturbation of the file. Prior to hashing, perturb the file and submit both the hash and the selected perturbation to apple. Apple selects the DB based on the perturbation and proceeds with matching and thresholding.

Re:Apple’s new abuse prevention system: an antritust/competi... by skinkestek on Saturday August 07, 2021 @12:50PM (#28099980)
> The discussion I thought we were having was about false positives and not adversarially induced false positives.

I think the rest of us have been discussing how this can and will be abused, by definition by adversaries.

Many of us have also observed for years how systems are abused so we sadly have a gut feeling for this.

Re:Apple’s new abuse prevention system: an antritust/competi... by Zababa on Saturday August 07, 2021 @12:20AM (#28095659)
I thought that the random collision for md5 was way higher than that. If it's that low, you're right, this would work. I'm not sure I understand the part about the pertubation.
Re:Apple’s new abuse prevention system: an antritust/competi... by DSingularity on Saturday August 07, 2021 @10:07AM (#28098691)
It’s actually 1/(2^128).

If the attacker does not know how the image will be perturbed prior to hashing then he cannot generate an image which matches with known CSAM.