News for hackers. Stuff that matters.

A comment on Convert Apple NeuralHash model for CSAM Detection to ONNX. Parent comment

Re:Convert Apple NeuralHash model for CSAM Detection to ONNX by shuckles on Wednesday August 18, 2021 @05:31AM (#28219155)
It might be useful to read the threat model document. Associated data from client neural hash matches are compared with the known CSAM database again on the server using a private perceptual hash before being forwarded to human reviewers, so all such an attack would do is expose non-private image derivatives to Apple. It would likely not put an account at risk for referral to NCMEC. In this sense, privacy is indeed preserved versus other server scanning solutions where an adversarial perceptual hash (PhotoDNA is effectively public as well, per an article shared on HN) would trigger human review of all the account’s data.
[ Parent | on HN ]
Re:Convert Apple NeuralHash model for CSAM Detection to ONNX by y7 on Wednesday August 18, 2021 @06:39AM (#28219505)
I assume these "non-private image derivatives" are downscaled versions of the original image. But for downscaling there also are adversarial techniques: perturb an image such that the downscaled version looks like a target image. See https://bdtechtalks.com/2020/08/03/machine-learning-adversar...
[ Parent | on HN ]
Re:Convert Apple NeuralHash model for CSAM Detection to ONNX by shuckles on Wednesday August 18, 2021 @06:42AM (#28219531)
For the server side component, the challenge is knowing the target of adversarial generation since, again, the perceptual hash used is secret. At that point, you are reduced to the threat present in every other existing CSAM detection system.
[ Parent | on HN ]
Re:Convert Apple NeuralHash model for CSAM Detection to ONNX by y7 on Wednesday August 18, 2021 @06:48AM (#28219560)
> the perceptual hash used is secret

Yes, although I'm sure a sufficiently motivated attacker can obtain some CSAM that they are reasonably sure is present in the database, and generate the NeuralHash themselves.

> At that point, you are reduced to the threat present in every other existing CSAM detection system.

A difference could be that server-side CSAM detection will verify the entire image, and not just the image derivative, before notifying the authorities.

[ Parent | on HN ]
Re:Convert Apple NeuralHash model for CSAM Detection to ONNX by zepto on Wednesday August 18, 2021 @10:57AM (#28221908)
> Yes, although I'm sure a sufficiently motivated attacker can obtain some CSAM that they are reasonably sure is present in the database, and generate the NeuralHash themselves

Remind us what the attack is here? The neural hash and the visual derivative both have to match for an image to trigger detection.

[ Parent | on HN ]
Re:Convert Apple NeuralHash model for CSAM Detection to ONNX by shuckles on Wednesday August 18, 2021 @11:34AM (#28222671)
I believe something like: an adversary creates an innocuous photo where:

* The photo itself is benign.

* The photo’s NeuralHash matches known CSAM.

* The photo’s image derivative is not benign. It looks visually like CSAM.

* The photo’s image derivative matches known CSAM per a private perceptual hash.

The above, combined, could have a victim reported to NCMEC without being aware they were targeted. Since Apple operates on image derivatives, they could be fooled unlike other cloud providers. That is the claim.

At that point, the victim could point law enforcement to the original CloudKit asset (safety vouchers include a reference to the associate asset) and clear their name. However, involving law enforcement can always be traumatic.

[ Parent | on HN ]
Re:Convert Apple NeuralHash model for CSAM Detection to ONNX by zepto on Wednesday August 18, 2021 @12:07PM (#28223255)
Sure, but did I miss something that suggests that all 4 of those conditions can actually be met? That seems like the part that has been made up.

Afaik the image derivative isn’t checked for looking like CSAM. It’s checked for looking like the specific CSAM from the database.

[ Parent | on HN ]
Re:Convert Apple NeuralHash model for CSAM Detection to ONNX by shuckles on Wednesday August 18, 2021 @12:57PM (#28223992)
CSAM in the database is confidential, so the human reviewer just needs to be plausibly confident that they have CSAM in front of them. However, it’s not clear to me that you can pull off all three simultaneously. Furthermore, the attack doesn’t specify how they would adversarially generate an image derivative for a secret perceptual hash that they can’t run gradient descent on.

If someone wanted to plant CSAM and had control of an iCloud account, it seems far easier to send some emails with those images since iCloud Mail is actively scanned and nobody checks their iCloud Mail account, especially not the sent folder.

[ Parent | on HN ]
Re:Convert Apple NeuralHash model for CSAM Detection to ONNX by zepto on Wednesday August 18, 2021 @01:06PM (#28224165)
CSAM in the database is confidential.

The question is whether the visual derivatives are checked against derivatives from the database or just against abstract criteria. That seems to be an unknown.

> However, it’s not clear to me that you can pull off all three simultaneously

Agreed. People here seem to keep assuming that you can, but so far nobody has demonstrated that it is possible.

[ Parent | on HN ]
Re:Convert Apple NeuralHash model for CSAM Detection to ONNX by nlitened on Wednesday August 18, 2021 @06:09AM (#28219323)
> before being forwarded to human reviewers

Does that mean that Apple employs people who manually review images known to be child pornography 9-to-5? Is it legal?

[ Parent | on HN ]
Re:Convert Apple NeuralHash model for CSAM Detection to ONNX by shuckles on Wednesday August 18, 2021 @06:25AM (#28219434)
Yes, and so does every other major cloud service provider. The working conditions of these people are notoriously difficult and should be the subject of attention.
[ Parent | on HN ]
Re:Convert Apple NeuralHash model for CSAM Detection to ONNX by kemayo on Wednesday August 18, 2021 @10:13AM (#28221148)
I'd imagine the key is that it's "manually review images suspected to be child pornography". The point of the review process is presumably that there are possible hash-collisions / false-positives, so the reviewers are what cause that transition from suspected to known.
[ Parent | on HN ]
Re:Convert Apple NeuralHash model for CSAM Detection to ONNX by TechBro8615 on Wednesday August 18, 2021 @11:04PM (#28229884)
Privacy is preserved by a team of humans looking at your private photos. Sounds like a good deal. What are we preserving again?
[ Parent | on HN ]