1. There can be false positives or other mechanisms for innocent people to get flagged.

2. It is bad to do this sort of check on the local disk.

The discussion at hand started as entirely #1. But now you've swapped to #2, talking about government spying on local files. It makes it very difficult to have a conversation because any pushback against arguments made for one point is assumed to be pushback against arguments made for the other point.

I am mostly convinced based on the technical details that have (slowly) come out from Apple that they have made this system sufficiently inconvenient to use as a direct surveillance system by a malicious government.

Yes a government could secretly order Apple to make changes to the system, but they could also order them to just give them all of your iCloud photos and backups, or send data directly from the Camera or Messages app, or any number of things that would be easier and more useful for the government. If you don't trust your government don't use a mobile device on a public cell network or store your data on servers.

But all of that said there is a still a line being crossed on principle and precedent for scanning images locally on device and then reporting out when something "bad" is found.

Apple thinks this is more private than just directly rifling through your photos in iCloud, but I can draw a line between what is on my device and what I send to Apple's servers and be comfortable with that.

And please do not bring in whataboutist arguments like "but other cloud providers are already doing it".

I'm honestly taken aback by how many people on HN are completely OK with what Apple is pulling here.

In my mind, it's irrelevant that the current system is "sufficiently inconvenient to use as a dragnet surveillance system", because it's the first step towards one that is convenient to use and if we extrapolate all the other similar efforts, we know full well what is going to happen.

And my point is that if the government wanted a dragnet they could just legislate or secretly order one. Just like they have done in various forms over the last 20 years. And Apple might not even be allowed to tell us it is happening.

More to the point, anyone remotely sophisticated can just encrypt CSAM into a binary blob and plaster it all over the cloud providers servers.

Ie, this system will possibly catch some small time perverts at the cost of even more potential of government misuse of the current technical landscape.

I consider it a similar situation to governments trying to legislate backdoors into encryption. A remotely sophisticated baddie will just ignore the laws, and all it does is add risks for innocents.

> And my point is that if the government wanted a dragnet they could just legislate or secretly order one. Just like they have done in various forms over the last 20 years. And Apple might not even be allowed to tell us it is happening.

I don't understand how is this an argument against the pushback at all. So just because it could be worse, we should just throw our hands in the air and say "Oh it's all fine, it could be worse so the (so far) mild intrusion into privacy is nothing to worry about."

The thing is, these things seem to happen step by step so the outrage is minimized. Insert your favorite "frog being boiled slowly" anecdote here. You don't push back on the mild stuff, and before you realize things have gotten so much worse.

I’m saying if the concern is that a government orders Apple to change it and do something different, then that’s a government problem and maybe we should try fixing that.

But why even give governments hints that people are generally OK with their devices being scanned?

We can argue about the technicalities of how abusable or resilient the current implementation is. But we can agree that it's a step towards losing privacy, yes? We didn't have scanning of iDevices before, now we do.

Because in my mind it's not a long shot to argue that once it becomes normalized that Apple can scan people's phones for CSAM when uploading to iCloud, it's just a small extra step to scan all pictures. The capability is basically in place already, it's literally removing a filter.

And then the next small step is not just CSAM but any fingerprint submitted by LE. And so it goes.

Governments can't legally compel Apple to implement this capability. But if the capability is already there, Apple can be compelled to turn over the information collected. Again, they can't do that if the capability and information doesn't exist.

E.g. if Apple can't decrypt data on your phone because they designed it such, then they can't be forced, even with a warrant, to backdoor your phone. They can legally refuse to add such capabilities.

Yes, that was the intention of my original comment. The “slippery slope” is one of principle and precedent, more than this specific technical implementation.

#2 is bad on it's own and in my mind it shouldn't even get to the merits of discussing #1 because at that point the debate is already lost in favor of surveillance.

But beyond that, #1 is highly problematic, doubly so given the fact that government surveillance is basically a non-decreasing function.