News for hackers. Stuff that matters.
Re:Apple enabling client-side CSAM scanning on iPhone tomorrow by headmelted on Thursday August 05, 2021 @03:56AM (#28071032)
This is the big one right here.

A malware will definitely be created, almost immediately, that will download files that are intentionally made to match CP - either for the purposes of extortion or just watching the world burn.

I'm usually sticking my neck out in defence of more government access to private media than most on HN because of the need to stop CP, but this plan is so naive, and so incredibly irresponsible, that I can't see how anyone with any idea of how easy it would be to manipulate would ever stand behind it.

Re:Apple enabling client-side CSAM scanning on iPhone tomorrow by dredmorbius on Thursday August 05, 2021 @05:39AM (#28071555)
Signal famously implemented, or at least claimed to implement, a rather similar-sounding feature as a countermeasure against the Cellebrite forensics tool:

https://signal.org/blog/cellebrite-vulnerabilities/

Re:Apple enabling client-side CSAM scanning on iPhone tomorrow by nceqs3 on Thursday August 05, 2021 @09:47AM (#28073665)
What is file that they have installed?
Re:Apple enabling client-side CSAM scanning on iPhone tomorrow by CRConrad on Thursday August 05, 2021 @06:01PM (#28080558)
If they told that, people would (try to) remove it. The whole point is that you can't know which (if any) of the hundreds of thousands of files on your device it is. So they aren't telling. Could be they have (or at least claim to have) written their system so it chooses files at random; I think that's what I would do (or claim to have done).
Re:Apple enabling client-side CSAM scanning on iPhone tomorrow by shuckles on Thursday August 05, 2021 @11:17AM (#28074931)
If this was easy to do, it’d already be a problem because Apple is already scanning some iCloud services for CSAM per their terms of service.
Re:Apple enabling client-side CSAM scanning on iPhone tomorrow by andrewmd5 on Thursday August 05, 2021 @08:34AM (#28072802)
If you can recreate a file so it’s hash matches known CP then that file is CP my dude. The probability of just two hashes accidentally colliding is approximately: 4.3*10-60

Even if you do a content aware hash where you break the file into chunks and hash each chunk, you still wouldn’t be able to magically recreate the hash of a CP file without also producing part of the CP.

Re:Apple enabling client-side CSAM scanning on iPhone tomorrow by ornornor on Thursday August 05, 2021 @09:25AM (#28073410)
The Twitter thread this whole HN thread is about shows just how to make collisions on that hash. So any image can be manipulated to trigger a match, even if that image isn’t CP.
Re:Apple enabling client-side CSAM scanning on iPhone tomorrow by ok123456 on Thursday August 05, 2021 @11:50AM (#28075428)
It's NOT a cryptographic hash.

It's the weights from the middle of a neural network that they're calling a "hash" because it encodes and generates an image it has classified as bad. Experts have trouble rationalizing about what weights mean in a neural network. This is going to end badly.

Re:Apple enabling client-side CSAM scanning on iPhone tomorrow by headmelted on Friday August 06, 2021 @06:50AM (#28085425)
Exactly this.

If this was a hash then it would be as the parent describes, this is at best a very fuzzy match on an image to take into account blurring/flipping/colour shifting.

It's vastly more likely that innocent people will be implicated for fuzzy matches on innocuous photos of their own children in shorts/swimming clothes than it is to catch abusers.

The other thing is, when you have nothing to hide you won't take efforts to hide it - meaning you'll upload all of your (completely normal) photos to iCloud without thinking about it again.

The monsters making these images know what they're doing is wrong, so they'll likely take efforts to scramble or further encrypt the data before uploading.

tldr; it's far likelier that this dragnet will only even apply to innocent people, than it is to catch predators.

All this said, I'm still in support of Apple taking steps in this direction, but it needs far more protections put in place to prevent false positives than this solution allows. A single false accusation by this system, even if retracted later and rectified, would destroy an entire family's lives (and could well cause suicides).

Look what happened in the Post Office case in the UK as an example of how these things can go wrong - scores of people went to prison for years for crimes they didn't commit because of a simple software bug.

Re:Apple enabling client-side CSAM scanning on iPhone tomorrow by dragonwriter on Friday August 06, 2021 @06:59AM (#28085486)
> The monsters making these images know what they're doing is wrong, so they'll likely take efforts to scramble or further encrypt the data before uploading.

The ones that make national news from big busts do, because the ones that don't get caught much sooner and only make local news, because Google and other parties are have automatic CSAM identification online already (server side, not client side, AFAIK), and are sending hits to Homeland Security.

Re:Apple enabling client-side CSAM scanning on iPhone tomorrow by saagarjha on Thursday August 05, 2021 @10:11AM (#28073978)
Adversarial examples are anything but random.