Furthermore, since the end is compromised, it's no longer a true end. The true end is the user and a system compromised in this way doesn't offer any guarantees to the user.
What is actually bad about their plan is the potential for scope creep and might wider scanning in the future. Your own, handwritten messages and personal photos can be secure even while this destroys many freedoms and lives.
Edit: thanks dpkonofa, I made the description vague in the hope that it's more correct.
Personally I've been dying for the next gen of M1 macs to come out. I also wonder if this tech has some magical way of getting around a hosts file and little snitch. If I can't gap my data from Apple it would raise ethical issues with my storing client secrets on the machine.
I don't think that's exactly where the "one in a trillion" claim comes from. Rather, it's that a single matching hash isn't enough to trigger the reporting; there needs to be multiple matches, and when there are enough of them to cross an unspecified threshold, then the reporting is triggered. There's theoretically only a one in a trillion chance of that threshold being crossed without having actual CSAM matches.
If I understand the white paper correctly, this even goes a step farther than that; they can't decrypt the signatures of the images corresponding to the matched hashes until the threshold is passed, because those images form a kind of decryption key together.
On a technical level, I'm actually pretty impressed. They absolutely could set up E2E encryption and still implement this system, and it largely assuages my worries about false matches of innocent photos (with the extremely big caveat that a false match has a very high potential of ruining someone's life). As the linked article points out, though, the real privacy concern here comes from having this matching capability on-device at all, because once it's there, limiting the data set to just this one provided by NCMEC becomes a matter of company policy. If an agency of any government demands Apple add their data set, they can no longer say, "we can't do that without drastically compromising the way our devices and services work," because it will be public knowledge that this in fact how their devices and services work already.
> Facebook is one of the biggest social media services. Back in 2013, they were receiving 350 million pictures per day. However, Facebook hasn't released any more recent numbers, so I can only try to estimate. In 2020, FotoForensics received 931,466 pictures and submitted 523 reports to NCMEC; that's 0.056%. During the same year, Facebook submitted 20,307,216 reports to NCMEC. If we assume that Facebook is reporting at the same rate as me, then that means Facebook received about 36 billion pictures in 2020. At that rate, it would take them about 30 years to receive 1 trillion pictures.
Bad cop: I want access to all your information.
Good cop: Oh, these guys are after you. Let me see if I can intervene for you. I can implement a selective limited search, that make these bureaucrats happy and at the same time will protect you. I am on your side, I hate them too. This is the best practical solution.
Ben Thompson's suggested approach for them in this article is to just do the unencrypted iCloud model with normal CSAM reporting like FB does and make that the tradeoff.
Apple actually wants things to be more secure by encrypting everything e2e with this CSAM hash approach to still fulfill that obligation (and protect themselves from future regulatory requirements that are more broad).
Reasonable people can disagree on these approaches to policy, but it's not obvious that Apple's move here is strictly worse for end users - if they do make moves to e2e then it's arguably better for user privacy to do it the way they're doing it.