And why would the NCMEC give law enforcement info about people they know are innocent?
For unencrypted content uploaded to their servers. That makes it a "Them Problem". Apple's approach is to scan before it is uploaded (and like the article says, "for now" - this can change with only a configuration change for all we know). And this could be a per-jurisdiction option and again, no one would know.
You have a point that this is not completely novel problem but do we have actual data on the number of real vs. false positives on the "everyone but Apple" group? Matthew Green probably would know but he doesn't[0].
[0] https://twitter.com/matthew_d_green/status/14283829528580833...
The left side photos are all apparently kitten photos but what if it was a photo of the Nirvana baby and, as a human reviewer, you had no access to the right side photo because it was just a hash in an anonymous database?
After all, Youtube copyright strikes have never been abused thanks to the fact that there are human reviewers.
Also, no app has ever been incorrectly removed from the Apple App Store, for the same reason that there are human reviewers.
This is thanks to the fact that human reviewers never make mistakes, unlike the rest of humans.
What could go wrong if we adopt the same system for something that could potentially result in a criminal investigation?
Are you able to explain why this specific case would be different?
They do report you to a private organization who is primarily funded from the US Government and has zero interest in not passing along any information about potential child abusers.
If you believe Apple, there will be a human review element. What sort of people would go after this kind of job? I can't imagine. And this group will also have a huge incentive to err on the side of reporting any potential abuse content.
Edit: A legitimate, almost definitely "wants to the the right thing" private organization in the US. Have we heard about who gets notified in other jurisdictions?