News for hackers. Stuff that matters.
Re:A catalog of naturally occurring images whose Apple Neura... by FabHK on Wednesday August 25, 2021 @08:32AM (#28300818)
So, the presumed attack (not against individuals, but to defeat the system) is

1. Identify some innocuous pictures that many many people have (memes, Beyoncé, whatever).

2. Produce CSAM.

3. Mangle it such that it is still CSAM visually, but NeuralHash-collides with the innocuous pictures from step 1.

4. Distribute.

5. Wait until they are (via some other mechanism) a) identified as CSAM, b) added to the NCMEC database, c) added to the Apple on-device database of blinded hashes in some iOS update.

6. Millions of people are suddenly incorrectly flagged for exceeding the threshold by NeuralHash (since they have the innocuous pictures in their library), and the review teams are flooded and can't pick out the small number of actual CSAM holders.

That is not without a certain elegance. However, it seems to me that

A) it is predicated on the assumption that you can easily mangle pictures to NeuralHash-collide with a desired target picture (out of a set of widely circulating innocuous pictures) without deteriorating the visual content too much.

B) it would be quickly defeated by amending the 2nd tier algorithm (between NeuralHash and human review), though, as you highlight, that might be tricky given that the team working on this presumably only has access to the innocuous false positive collision image, not the (purposefully mangled) CSAM.

Re:A catalog of naturally occurring images whose Apple Neura... by saithound on Wednesday August 25, 2021 @08:46AM (#28300926)
> A) it is predicated on the assumption that you can easily mangle pictures to NeuralHash-collide with a desired target picture (out of a set of widely circulating innocuous pictures) without deteriorating the visual content too much.

Note that this requires no single "desired" target picture. There are millions of popular, innocuous pictures. As long as you can make your CSAM match any one of them without significant mangling, you're good to go. Not having to choose one specific target makes this much easier to accomplish.

Re:A catalog of naturally occurring images whose Apple Neura... by nullc on Wednesday August 25, 2021 @02:27PM (#28305107)
> it is predicated on the assumption that you can easily mangle pictures to NeuralHash-collide with a desired target picture (out of a set of widely circulating innocuous pictures) without deteriorating the visual content too much.

You can. Here is an example I created (with links to more): https://github.com/AsuharietYgvar/AppleNeuralHash2ONNX/issue...

I'm so tired of people suggesting that you can't. Please explain to me why you posted suggesting otherwise.

I've contemplated making some that are also photodna matches, I expect that it's possible. But access to photodna is only through some awful windows tools, and AFAICT people would just keep posting denials even after an example was posted-- so it's not worth the effort at least not worth it just to further the public discussion.

Re:A catalog of naturally occurring images whose Apple Neura... by FabHK on Wednesday August 25, 2021 @05:27PM (#28307268)
> Please explain to me why you posted suggesting otherwise.

a) I didn't suggest otherwise, I said that it is predicated on that assumption, about which I was undecided, largely because b) I didn't know better.

I read that thread 7 days ago, when the collisions were a gray blob or a clearly modified dog (to Lena) or clearly modified Lena (to dog). I hadn't re-read the thread in the last 4 to 5 days, when you demonstrated the natural looking collisions (second-preimage images).

Very impressive work I wasn't aware of.

Re:A catalog of naturally occurring images whose Apple Neura... by nullc on Wednesday August 25, 2021 @05:34PM (#28307353)
Great answer. Thank you!
Re:A catalog of naturally occurring images whose Apple Neura... by SiempreViernes on Wednesday August 25, 2021 @09:05AM (#28301135)
There's also the _very_ important assumption that Apple doesn't check for collisions with popular images before adding new things to the database.