News for hackers. Stuff that matters.
Re:Apple kills plans to scan for CSAM in iCloud by lathiat on Wednesday December 07, 2022 @08:02PM (#33902721)
EDIT: Whether or not Apple should be scanning for CSAM in some way is an entirely separate argument to whether it should be "on device" or "in cloud". That's not obvious, so my explanation below explains why that is the case.

I disagree. On-device scanning is desirable here and scanning it in iCloud though the industry-norm, is not desirable.

Apple specifically stated they would only scan the photos being uploaded to iCloud [1] and not local photos which were not being uploaded.

Most cloud storage providers are scanning in the cloud, this necessitates a workflow where both the photos and decryption keys are accessible by the same server and that there is a security workflow to request the users decryption keys without the user involved (by way of their account password, or other enforcement mechanisms designed to ensure only the real user can access such keys, Apple has previously detailed Hardware HSM enforced details of that for iCloud in their Apple Platform Security guide [2]).

By extension that opens up the potential for a malicious employee or third-party to either intercept those photos while being scanned or otherwise allows a workflow that could be exploited to request the keys and photos. This is a major loss of security and privacy which Apple is specifically and intentionally designing to avoid. Until now iCloud Backups were a major weakness in that story however they also today announced they are fixing that [3].

Hence: the on-device scanning was just an entirely technical solution to scanning "only photos uploaded to icloud" (like everyone else) while also "ensuring some server in icloud can't decrypt your photos".

You could of course argue that it makes a later change of policy to scan local-only photos easier. While this is true, Apple could decide to add code to iOS to do anything they want at any time, so they could just decide to do that anyway, even if they were scanning iCloud photos in the cloud. At the same time, by not scanning them in the cloud, they have a technical solution to ensure it's much harder for a malicious actor to get unauthorised access to iCloud user photos. Because there is no place/server that is "supposed" to be able to access both the photos and the keys.

[1] https://www.apple.com/child-safety/pdf/Expanded_Protections_... [2] https://help.apple.com/pdf/security/en_US/apple-platform-sec... [3] https://news.ycombinator.com/item?id=33897793 "Apple introduces end-to-end encryption for backups"

Re:Apple kills plans to scan for CSAM in iCloud by nomilk on Wednesday December 07, 2022 @08:19PM (#33902850)
> On-device scanning is desirable

What are the benefits? It would tell Apple (and hence law enforcement) who had copies of existing, known CSAM. I guess that would help catch such people. But such individuals would also have strong incentive to simply turn of iCloud to completely avoid detection.

What are the risks? Apple employees rummaging through your photos (after 30 NeuraHash matches, a human reviewer would go in and check your photos, and as some analysts pointed out, hash matches can be spurious, that is, completely different photos can have the same hash by coincidence).

Over time, would the program expand beyond CSAM? Consider the intersection of countries in which Apple sells iPhones and countries that criminalise being LGBTQ [1]

[1] https://www.humandignitytrust.org/lgbt-the-law/map-of-crimin...

Re:Apple kills plans to scan for CSAM in iCloud by lathiat on Wednesday December 07, 2022 @08:24PM (#33902892)
All of your concerns are valid for "should Apple do CSAM scanning at all" and I share those.

However my argument is specfically to the OPs point that they should just scan the photos "in the cloud" and not "on your device". But the reality is they were only scanning photos that were being uploaded to the cloud, they were just doing it on your device so that Apple doesn't need to have a server that can decrypt your photos to do it in the cloud. Which is what most others (including Google) do.

Re:Apple kills plans to scan for CSAM in iCloud by JimDabell on Thursday December 08, 2022 @12:14AM (#33904460)
> as some analysts pointed out, hash matches can be spurious, that is, completely different photos can have the same hash by coincidence

I think you might have misinterpreted whatever those analysts wrote. The chances of thirty images all coincidentally matching both perceptual hashes is essentially nil.

Re:Apple kills plans to scan for CSAM in iCloud by theshrike79 on Thursday December 08, 2022 @02:57AM (#33905440)
The human reviewer can't "go in and check your photos".

The human reviewer would've gotten "reduced quality images" of the matches and would make a decision based on those.

And yes, people could've just turned off iCloud. That's how encryption works.

Re:Apple kills plans to scan for CSAM in iCloud by stjohnswarts on Wednesday December 07, 2022 @08:59PM (#33903134)
I can't agree. Normalizing scanning of files on people's devices is a very very bad precendent. It turns Apple into a low level government policeman rifling through your device which would spread to your PC and android devices. Normalizing this behavior would be truly awful, CSAM would come first, then tax audits, anything that looks "suspicious", etc. That is a very very bad precedent, police should only be able scan your device if they have a warrant.
Re:Apple kills plans to scan for CSAM in iCloud by YurgenJurgensen on Thursday December 08, 2022 @03:29AM (#33905664)
The normalisation aspect of this would have been the biggest horror. It would mean owning a device that doesn't do this becomes associated with criminality. "Don't you know that only pedophiles run Linux?"
Re:Apple kills plans to scan for CSAM in iCloud by ec109685 on Thursday December 08, 2022 @01:30AM (#33904875)
It wasn’t scanning photos looking for csam, it was computing a fingerprint of every photo that could only be read on the server if 5 of those fingerprints matched known csam.

I can’t see how that is worse than unencrypted photos in the cloud that can be scanned at any time server side.

Re:Apple kills plans to scan for CSAM in iCloud by stjohnswarts on Thursday December 08, 2022 @04:58PM (#33914199)
It's a proxy for scanning on your phone and reporting to the government. Toe-mae-toe toe-mah-toe
Re:Apple kills plans to scan for CSAM in iCloud by yyyk on Wednesday December 07, 2022 @08:23PM (#33902883)
If Apple were to calculate the perceptual hash client-side but only do the actual hash comparison server-side following an upload, it would preserve the 'no keys on iCloud' policy while also preserving the 'no local scanning' policy most of us want.

That the server may know the hash is IMHO much less a problem (it shouldn't be reversible anyway) than the legitimization of on-device scanning.

Re:Apple kills plans to scan for CSAM in iCloud by lathiat on Wednesday December 07, 2022 @08:35PM (#33902958)
I would think calculating the perceptual hash client side is precisely the "local scanning" part most are concerned about. Pushing a perceptual hash of every photo to Apple would actually decrease your privacy as now Apple can find any known photo in your library - not just CSAM but on any topic - memes, political protests, porn, whatever.
Re:Apple kills plans to scan for CSAM in iCloud by yyyk on Wednesday December 07, 2022 @08:49PM (#33903066)
IMHO, it's the comparison to a stored hash value which is the scanning part. In my scenario, the hash calculation is only done during upload and the hash is not saved on device.

Let's say tomorrow China tells Apple to do client-side scanning, and let's say Apple decides to comply.

Scenario A (Apple already did on-device scanning):

Apple flips a few bits. iOS no longer cares whether images where uploaded on not. The code change can easily be hidden assuming no reverse engineering. There's little measurable difference unless the user has a matching image in which case the user is already done for [EDIT: in retrospect, the device should be running a scan on the local images every blacklist fetch. I still think that scanning more images is much less of a difference than the other scenario, where the device has to do stuff it just did not do before]

Scenario B (Hash Comparison is server-side):

The code change can be hidden of course. But otherwise there are significant changes:

The device needs to calculate hashes to every image upon creation which it did not before. The device needs to fetch the blacklist every X days which it did not before. The device needs to run the local scan every blacklist fetch which can be slow if many images.

All these things are far more likely to be noticeable and far more likely to generate network traffic.

>Pushing a perceptual hash of every photo to Apple would actually decrease your privacy as now Apple can find any known photo in your library - not just CSAM but on any topic - memes, political protests, porn, whatever.

The Hash shouldn't be reversible, so the only way for Apple to do that is to have an existing dataset. Which is possible, but preferable to the no E2E situation and (IMHO) the on-device scanning situation.

Re:Apple kills plans to scan for CSAM in iCloud by ec109685 on Thursday December 08, 2022 @01:28AM (#33904866)
That’s not how their encryption worked. The only way for apple to know a csam photo matched was if you had 5 photos that had a csam fingerprint from their database uploaded to iCloud. This matching only happened server side for photos uploaded to the cloud.

This seems far better than having every photo scanned server side and have those photos sit there unencrypted.

Re:Apple kills plans to scan for CSAM in iCloud by brookst on Wednesday December 07, 2022 @08:39PM (#33902988)
I think people objected to having the flagged hash database and comparison locally. I suppose the split of perceptual hash client side, comparison server-side might be objectionable on the "it's draining my battery" angle, but I suspect those who prefer cloud-side scanning would generally like hash creation client-side if it were the price of E2EE.

Of course that's a lot less secure and leaks more privacy info, but the objections seem more about the morality of having a device you own doing the scanning, not so much about any privacy concerns.

Re:Apple kills plans to scan for CSAM in iCloud by ec109685 on Thursday December 08, 2022 @01:57AM (#33905049)
The comparison didn’t happen locally in Apple’s scheme. Along with each photo a encrypted fingerprint was uploaded and all scanning happened sever side.
Re:Apple kills plans to scan for CSAM in iCloud by brookst on Thursday December 08, 2022 @11:56AM (#33910153)
That wasn't my read from https://www.apple.com/child-safety/pdf/CSAM_Detection_Techni... :

Lots to look at there but the key bit is:

>If the user image hash matches the entry in the known CSAM hash list, then the NeuralHash of the user image exactly transforms to the blinded hash if it went through the series of transformations done at database setup time. Based on this property, the server will be able to use the cryptographic header (derived from the NeuralHash) and using the server-side secret, can compute the derived encryption key and successfully decrypt the associated payload data

So my understanding is that the server won't even know the hash of the image unless the client-side operations determine that it matches one of the hashes in the CSAM database, which is stored on the client.

Are you seeing it differently?

Re:Apple kills plans to scan for CSAM in iCloud by ec109685 on Friday December 09, 2022 @10:26AM (#33922258)
I’m seeing the same. There are a lot of people saying apple’s scheme was somehow worse than Google’s. I was arguing that apples was at least attempting to be privacy preserving.
Re:Apple kills plans to scan for CSAM in iCloud by karulont on Thursday December 08, 2022 @01:34AM (#33904898)
Perceptual hashes are quite reversible. It is not a cryptographic hash that is hard to reverse.
Re:Apple kills plans to scan for CSAM in iCloud by yyyk on Thursday December 08, 2022 @06:17PM (#33915139)
I think the state of the art has progressed past that? A non-trivial reversal of a perceptual hash would mean that every cloud provider maintaining a CSAM scan list violates CSAM laws - if they could reverse the hash to get too close to the original image, the hash is just a lossy storage format.
Re:Apple kills plans to scan for CSAM in iCloud by ec109685 on Thursday December 08, 2022 @01:55AM (#33905039)
Not the way apple implemented it. The hash was encrypted and could only be decrypted if 5 hashes matched a known csam hashes.
Re:Apple kills plans to scan for CSAM in iCloud by TreeRingCounter on Wednesday December 07, 2022 @10:07PM (#33903646)
> On-device scanning is desirable

My own devices should not snitch on me to the government. Obviously this mandatory snooping of all private data is not going to stop at the single most politically salable application; that's just the starting point.

Re:Apple kills plans to scan for CSAM in iCloud by formerly_proven on Wednesday December 07, 2022 @08:18PM (#33902841)
iOS doesn't have local-only photos. You either put all photos in iCloud, or none (bar iCloud being full).
Re:Apple kills plans to scan for CSAM in iCloud by lathiat on Wednesday December 07, 2022 @08:20PM (#33902856)
You are right, but by local-only photos I meant "not using iCloud photos" - e.g. "none".

The point is, I think if Apple decided to scan your photos for CSAM even if you weren't using iCloud, that this would be a distinctly different situation to scanning only the ones uploaded to iCloud.