Eventually it could lead to Apple platform not having any CSAM content, or any known legally objectionable content, because any sane perpetrator would migrate to other platforms, and less sane, caught.
This, of course, takes the user base to overwhelmingly agree that keeping legally objectionable content is a bad thing and should be exposed to police, and to trust the whatever body which defines the hashes of objectionable content. I'm afraid this order is not as tall as we could imagine.
And yet photos that get scanned are still uploaded to iCloud Photos, so they do end up on Apple's servers.
Doing so right while activating a new iDevice is the way to prevent its private keys from ending up in iCloud, and so preventing Apple, or law enforcement, or some malicious hackers from cracking into your device.
I'm failing to see the issue.
The results of the scan looking for kiddie porn cannot be read by Apple until the device finds 30 examples of photos that match known kiddie porn, whereupon Apple gets the decryption key so they can see which images on their server need review, and a human review is triggered to make sure there haven't been 30 false positives.
I implore you to read the comment I originally replied in order to understand the context of my replies. Personally, I don't care what you're shocked about or not, as my OP wasn't directed at you at all.
After that, they get the decryption key and trigger a human review to make sure there haven't been 30 false positives at once.
That is better, and more private, in a very meaningful way.
False positive scan data sitting on the server is open to malicious misuse by anyone who can get a warrant.
>Innocent man, 23, sues Arizona police for $1.5million after being arrested for murder and jailed for six days when Google's GPS tracker wrongly placed him at the scene of the 2018 crime
https://www.dailymail.co.uk/news/article-7897319/Police-arre...
Also, I sincerely doubt that Google has ended it's practice of refusing to hire a human being to check for false positives when a poorly performing algorithm is cheaper.
I'm fine with the idea that if I upload stuff to someone else's server, they may take a look at it and maybe even punish me for what I've uploaded. Certainly if I encrypt the data before I upload it, they can't do that. But if I don't, then it's fine with me if they do.
But my device should not be snitching on me. Yes, this device-side scanning is supposedly gated on enabling uploads to iCloud, but that doesn't really make for much of a distinction to me. And since Apple certainly has the capability, they are likely one secret court order away from being required to scan even photos that aren't being uploaded, at least on some targeted subset of devices.
Apple has the ability to upload literally any software to iPhones, so this argument applies equally to literally any conceivable bad thing that software could do on iPhones.