A lot of the contention wasn't about the specifics of their plan, but rather how subtle changes could vastly expand the scope of their plan.
"this would only apply to photos sent to iCloud." for now, until scope creeps.
"this was only looking for previously known CSAM." for now, until scope creeps.
"using some sort of hash matching so photos you took yourself would not trigger the system." well this one is immediately concerning even within claimed scope because there ARE going to be false positives that apple records some database. Millions of iphone users are going to have a non-zero "possible childporn" score.
They are building an engine for iphone users to self-incriminate. If they rigidly hold the scope to only what they announced and never expand, it could be argued that this is a reasonable concession to fight CSAM. However, in making the announcement, they boldly stepped past their existing hard line in privacy (local device content is private and not surveilled by apple), so it seems naive to expect that this announcement reflects the eventual scope of this self-incrimination engine for the next decade of apple updates.
I don't think this is right. The apple pdf on the features says
> With the initial match threshold chosen as described above, iCloud Photos servers learn nothing about any of the user's photos unless that user's iCloud Photos account exceeded the match threshold.
This implies to me that they are using some sort of encryption to prevent iCloud from learning even how many matches there are until the threshold is met.
The how helps show us how changing this system is not a subtle change. It isn't like they can flip a switch and suddenly they are identifying new suspected CSAM on people's phones. That would require a new system since the current one is only hash matching.
>However, in making the announcement, they boldly stepped past their existing hard line in privacy (local device content is private and not surveilled by apple), so it seems naive to expect that this announcement reflects the eventual scope of this self-incrimination engine for the next decade of apple updates.
This is an arbitrary line that is being drawn. These are photos that are marked for sending to iCloud. Whether the scanning happens on the phone before they are sent or in the cloud after they sent is largely immaterial when it comes to the impact of the code. People are acting as if the line Apple drew was motivated by technology. That was never the deciding factor. Technology is the easy part here. That line was only a policy line and that policy has not changed. Only photos that are sent to iCloud are scanned. If you fear Apple changing that policy going forward, you should have always feared Apple changing that policy.
This is a strawman. Identifying *novel* CSAM is a very hard problem to do accurately - the reason they can't flip the switch is because they don't have the technical capability. All of the other things people are concerned about are things that apple does have the capability to do.
EDIT to reply since at max thread depth: *novel* image detection was never on the table, I think you missed that word
> This is an arbitrary line that is being drawn.
It seems the vast majority of people in this thread disagree that this line is arbitrary.
EDIT: whether the threshold is "verging on impossible." depends entirely on the effective false positive rate. If apple's claimed 1 in 1 trillion rate is true, it's probably not a concern. However, I find it unlikely that perceptual hashes on portions of images won't have higher false positive rates when subject matter is similar (non-CSAM legal adult porn, or images of children in swimsuits, etc). If that rises to 1 in 1 million for these types of images, that's hundreds of thousands of people being falsely accused.
You just used this as an argument against this system. Why do you fear this if you don't think Apple can even accomplish this technologically?
>It seems the vast majority of people in this thread disagree that this line is arbitrary.
I would argue that people who believe that this decision crossed that line were being naïve to not have always known this was a possibility. I don't think this move brings us any closer to Apple scanning our devices for anti-government memes or whatever the fear is because what was stopping that was always more policy than technology.
Also to go back to your earlier comment, in the time since you posted it has now been revealed that this system needs to trigger 30 times before any action is taken. The odds of 30+ false positives is likely verging on impossible.
I'm a huge Apple fan and a Mac devotee of 30+ years. I love my iPhone. If this thing becomes real, I will go shopping for another phone.
This is absolute insanity. I can't believe they even thought about launching this.
I'm completely disgusted.
Maybe. I wonder how many people are choosing not to respond because they’ll only be voted down by the people who feel very strongly about this. I’ve curtailed and hedged many my contributions to this topic on HN because of this.
This is a common issue when social media is used to debate matters which are highly emotionally asymmetric.
* It was not universally understood that this would only apply to photos sent to iCloud.
Since the scanning doesn't happen on iCloud, this distinction is irrelevant.
"We are going to intrusively scan the subset of your photos that you care enough to back up to the cloud that we've been pushing to you for years" is pretty clear.
* It was not universally understood that this was only looking for previously known CSAM.
It was only looking for whatever is in an opaque database which, according to a third party we don't have any contract with, contains CSAM.
* It was not universally understood that they were using some sort of hash matching so photos you took yourself would not trigger the system.
Yeah right, I feel totally safe knowing that I won't be falsely reported to FBI by a "some sort of" hash matching.
Here's a hash function: f(x) = 0 for all x
It's "some sort of" hash, too.
It’s really not irrelevant. A third party photo library that avoids using the PhotoKit library would not be touched by the CSAM detector. There are many of these on the App Store.
One step further and store the photos encrypted, with a custom renderer that decrypts the content on the heap, and that would take some tremendous performance-hitting detection abilities it’s extremely unlikely to ever happen.
OK.
Misrepresenting each other’s arguments is not how you have a discussion.
I hope that's precise enough, and sure, everyone is welcome to make their own decisions regarding this.
- Whose fault is it that those points were not clearly communicated?
- Who wrote the perceptual hash matching code?
- Who is allowed to audit the code, the review system, and the hash database?
- Who updates this code?
- Who decides if your phone OS is updated?
- Who decides the iCloud upload defaults?
- Who decides if you are reported?
- Who asked for this feature?
To the extent that other parts of this story was explained to us by Apple, I did try to clarify some exaggeration in other thread.
It was not universally understood that this would only apply to photos sent to iCloud.
It was not universally understood that this was only looking for previously known CSAM.
It was not universally understood that they were using some sort of hash matching so photos you took yourself would not trigger the system.
I understand if you consider the where more important than the others, but it is simply a fact that there was confusion on what exactly was happening here.