Still, I think Apple misjudged the whole cloud vs. device thing in this case. They’ve historically preached a lot about how everything should happen on the users device, not the cloud. I think that got myopic for them, and led them to this decision.

But in this case I think users would be much happier if Apple had just said “under pressure from law enforcement we are now scanning photos when they arrive at the iCloud data centers. If you don’t want scanning don’t use iCloud.” Because it’s not so much the scanning of uploaded photos that has people upset, it’s the fact that the scanning and phoning home is baked into the device itself.

Many people (like myself) are worried about the slippery slope where this is turned on for all photos, since why not? Not all abusers will upload their CSAM content to the cloud, why wouldn't Apple flip a flag in the future to scan everything, including photos and downloaded content? If they are serious about fighting CSAM and have this great privacy preserving platform, I don't see why they wouldn't do this?

The point of the feature is to prevent people from using iCloud to distribute CSAM. If you’re recording it with your phone, it’s no different than using an slr camera. The cloud part is what they’re worried about.

Right, which is why it's so utterly baffling that they don't do this scanning server-side instead of client-side.

edit: like many comments here already say, reporting doesn’t sound terrible for CSAM, but nothing about the feature guarantees it wont be extended to other kind of content.

That's for now. The first update can change that and you will have no recourse.

Also, the matches are supposedly only to actual babyporn pictures. We have 0% way to verify that, as even employees of NEMSEC are not all allowed to view them. Such DBs are often full of unreviewed fluff, and why not unrelated photos entirely, cookware, computer cases, who knows, as long as “some degree of matching” with your photos allows Apple to send a .zip to the police.

In terms of the privacy intrusion, what difference does it make whether the images are scanned on your device or on their servers? They're getting scanned just the same either way.

But if they did it on their servers, it would provide a technical impediment to expanding beyond the use of iCloud, and remove the need to trust Apple as much. That seems like it would be much better for users while still allowing the functionality they claim to be seeking.

What makes you say this? They announced this change after all. Why wouldn’t they announce future changes?

And I don't think that would include photos saved in other apps.

Apple promising not to use the scanner is a weak promise they know they can’t keep (NSLs)