> Steve Jobs must be rolling over in his grave.

June, 2010 - Jobs: "Privacy means people know what they're signing up for."

https://youtu.be/39iKLwlUqBo?t=154

December, 2015 - Cook: "If you put a back door in, that back door is for everybody, for good guys and bad guys."

https://youtu.be/rQebmygKq7A?t=66

August, 2021

Interviewer: "Isn't this in a way a back door?"

Federighi: "I really don't understand that characterization."

https://youtu.be/OQUO1DSwYN0?t=426

Apple really is talking down on people on that last one.

> I can't imagine anyone would do day-to-day testing with actual illegal images.

You select any random image as your test image, marking that image as CP in the backend and then try to upload that image from your test phone.

Understood that that is how you test in the early stages. But at some point you have to run tests against the actual deployed perceptual hash database.

No reason why you can't just generate random images and add them to the real database, indistinguishable from actual hashes. As long as the test images aren't public it's low risk, and even if they leak they could be removed from the database and new test images could be generated.

Wouldn't that mean that people created a system that can be used with normal pictures? So your code was literally to flag an innocent photo?

That makes sense. I would still quit the team.

Apple's "CSAM detection" feature is a two-part on-device/server as I understand it:

1: Run on-device code to perform perceptual hash comparison of each photo against an on-device encrypted database of known CSAM hashes.

2: On iCloud Photos servers, send out the relevant notifications when a user’s iCloud Photos account exceeds a threshold of positive matches.

So as a high level testing strategy, I would want to:

- Verify on-device lookup of CSAM hashes. This could be tested by provisioning a test device with an on-device database containing CSAM hashes of images that aren't illegal. As a bystander, I think I'd be fairly confident with this approach because I'm guessing the on-device database that Apple ships could conceivably be changed over time to expand the definition of the images it will flag as CSAM.

- Do some exploratory testing to discover the threshold of how much image manipulation can be done on a flagged image before the perceptual hash comparison fails to return a match.

- Verify that the notification system notifies the correct parties once a user account exceeds the defined threshold of positive CSAM matches.

- Ensure the flagged account can still be investigated if user deletes the offending material from iCloud, or their account by the time a real person gets around to investigating.

- Ensure that the logging is informative and adequate (contains device name, timestamp, etc.).

- Test behaviour on same iCloud account logged in to multiple devices.

- Figure out any additional business logic - are positive matches a permanent count on the account or are they reset after a certain amount of time?

source: https://www.apple.com/child-safety/pdf/Security_Threat_Model...

You wouldn't test this with child abuse images. You would likely test it with a set of legal images, for which you generated your own hashes.

>This is all awful. Steve Jobs must be rolling over in his grave.

They didn't even last 10 years since his death. :/