As recently posted on HN [1] one should be very wary of backdoors, no matter how much one believes only the right guys could ever use them. Once they're there, they're there: it's arrogant beyond believe to think that opponents of yours won't exploit them.
I won't use an apple product with this feature. I've been using apple's ecosystem since about 1994.
[1] https://twitter.com/matthew_d_green/status/14334701097425182... and https://news.ycombinator.com/item?id=28404219
I don't understand this, they can just be legally compelled.
But a motivated bad actor has a lot easier time just putting an image of the Taiwanese flag or propaganda on someone's phone than trying to make it a hash collision that triggers... If an attacker is really after someone, I would expect them to put the actual material on that person's phone...
... It's like trying to frame someone for drugs in their car and going through the hassle of synthesizing a chemical that triggers drug dogs to react, but it isn't actually the drug. Wouldn't they just... buy drugs and stick them in the car?
Maybe they want to create too many false positives deliberately, or maybe they do it just because they want to see if and how synthesizing such a chemical can be done.
When putting pictures on someone's phone or computer, there are ways to add false positives as well as maybe doing false negatives sometimes (e.g. by encrypting the picture or using an uncommon file format so that it must be converted, or using complicated HTML/JS/CSS to render it instead of just a picture file).
Also, if someone has the picture or drug or whatever to find if it is what they are, can you accuse them (maybe they are the police) of liking pictures and drugs, too?
But, to be clear, you make a legitimate point.
How so? Unlike other cloud providers, which do scan all uploaded images server side, this system is specifically designed to prevent this.
(As you say, if Apple is entirely compromised, then all bets are off anyway.)
I think there _is_ an argument to be made about a system like this being used to track the spread of political material, and it's easy to see how such a system would be terrible for anyone trying to hide from an authoritarian power of some type, but that'd already require Apple is absolutely and completely compromised by an authoritarian regime, which isn't high on my list of concerns.