Apple's "CSAM detection" feature is a two-part on-device/server as I understand it:

1: Run on-device code to perform perceptual hash comparison of each photo against an on-device encrypted database of known CSAM hashes.

2: On iCloud Photos servers, send out the relevant notifications when a user’s iCloud Photos account exceeds a threshold of positive matches.

So as a high level testing strategy, I would want to:

- Verify on-device lookup of CSAM hashes. This could be tested by provisioning a test device with an on-device database containing CSAM hashes of images that aren't illegal. As a bystander, I think I'd be fairly confident with this approach because I'm guessing the on-device database that Apple ships could conceivably be changed over time to expand the definition of the images it will flag as CSAM.

- Do some exploratory testing to discover the threshold of how much image manipulation can be done on a flagged image before the perceptual hash comparison fails to return a match.

- Verify that the notification system notifies the correct parties once a user account exceeds the defined threshold of positive CSAM matches.

- Ensure the flagged account can still be investigated if user deletes the offending material from iCloud, or their account by the time a real person gets around to investigating.

- Ensure that the logging is informative and adequate (contains device name, timestamp, etc.).

- Test behaviour on same iCloud account logged in to multiple devices.

- Figure out any additional business logic - are positive matches a permanent count on the account or are they reset after a certain amount of time?

source: https://www.apple.com/child-safety/pdf/Security_Threat_Model...