Why do you think somebody will accuse you of a crime because you have a photo of a grey blob?

The real world isn't as stupid as the computer one. The justice system is not deterministic and automatic. Nobody is going to look at this grey blob and go "welp, we have no choice but to throw you in prison forever"

The real world and the computer world intersect. This is precisely typified by what is being discussed, surely?

Apple is trying to automate and computerise a process that was not automated previously, apply it to a huge number of people, and with disastrous potential consequences should their wonderful design be found lacking.

And within days, they have already utterly failed to provide one of their own self-stated and incredibly obvious requirements. What else could possibly go wrong?

Further, if you think that because the first pre-image failure found (in days...) was a grey blob, that this means all future possible cultivated collisions will only ever be grey blobs, well - good luck with that.

The process of making accusations and arrests is not automatic.

The Apple system sends flagged photos to reviewers, and if the reviewers find them suspicious, they send them to the poor souls at NCMEC, who will compare the flagged photo with the original illegal photo that's supposedly a match, and inform law enforcement if they are in fact a match.

Nobody will get cops at their door because somebody sent them a grey blob image.

The process that's being "automated" is merely an automatic flag that initiates a several-step process of human review. Apple isn't rolling out robocops.

Seriously, what are the "disasterous consequences" that you envision? What is the sequence of events where a hash collision leads to any inconvenience whatsoever for a user?

What part of "they already totally fucked up how their own process is supposed to work" don't you understand?

What part of it is fucked up? They never promised that their hashing algorithm was uncollidable. The process is specifically designed to be tolerant of hash collisions (and in fact wouldn't work otherwise, because the system needs to ignore small differences between copies of the illegal images, like one-pixel edits or color temperature differences or photocopies).

There are multiple stages of human review in the process and a high threshold for activation of the proccess because collisions are inevitable. The fact that collisions exist doesn't impact the safety of the product whatsoever.

I ask again, what is the sequence of events where a hash collision from a benign image can lead to any consequence at all for the user?

"Apple said that there is a one in one trillion chance of a false positive"

https://techcrunch.com/2021/08/05/apple-icloud-photos-scanni...

Versus:

https://blog.roboflow.com/nerualhash-collision/

Along with:

https://github.com/anishathalye/neural-hash-collider

Demonstrates blatant incompetence on the part of Apple's design. At this point, trust is broken. That's the end of credibility regarding any other claim made about the security or base function of the system.

As for a simple example of one of many potential attack vectors (search HN for "neuralhash" and read the comments for countless examples), here is a basic attack:

As Apple's hash algorithm is entirely broken and useless, a malicious actor can easily craft a NSFW, but otherwise legal image, that has a hash matching one in the CSAM database, that is visually ambiguous in that context (for example, a close-up). Targeted attacks could be launched on unsuspecting victims as simply as sending them an image.

If their app is configured to auto-save to iCloud - as some are by default, and as many are configurable to be - the image will then get automatically flagged as a match, and may well potentially pass a human review - because the image may indeed look just like CP - and an innocent person, who may not even be aware of the presence of the image in their iCloud library at this point, may get visited by the police.

That in itself is bad enough.

However it gets worse - do bear in mind that "the police" is not an equivalent concept across nations, yet iPhones are ubiquitously used across the world. In certain countries, a "visit from the police" over such automatically-flagged content could well result in presumption of guilt to the point of immediately administered punishment and a destroyed innocent life.

In China for example, conviction rates are routinely 99.99% - being accused of a crime there is equivalent to being convicted, and that's just one major country containing well over 100 million iphone users.

Apple is demonstrating not only gross incompetence in this design of this "system", but vagrant disregard for human rights, along with utterly destroying their own long-cultivated pro-privacy stance.

It's a disaster for them and their customers in every possible way it could be.

Apple was talking about the chance of accidental false positives.

There is a 1 in 4 billion chance that if I send a message to an IP address, it will be yours. The fact that I can know your IP address and send you a message intentionally does not make the previous statement false.

> As Apple's hash algorithm is entirely broken and useless, a malicious actor can easily craft a NSFW, but otherwise legal image, that has a hash matching one in the CSAM database, that is visually ambiguous in that context (for example, a close-up). Targeted attacks could be launched on unsuspecting victims as simply as sending them an image.

Everybody keeps talking about this scenario as if it is any different whatsoever from sending people actual regular old-fashioned CSAM, which A: already happens, B: requires less effort on the part of the attacker, and C: isn't reliant on the recipient uploading them to iCloud, triggering several layers of filtering and detection, which may or may not result in law enforcement interest depending on your forgery's ability to fool professional investigators at NCMEC who are comparing it alongside the original.

The attack you're describing is less effective and more difficult than existing ones.

No, the attacker is committing a crime with the attack you describe, but has plausible deniability that they are not with the Apple-nonsense enabled attack vector.

This is a huge difference as not only does Apple facilitate a gain-of-function in terms of potential detriment to the victim of an attack, the newly-enabled vector also indemnifies the attacker.

The effort required to come up with a hash collision in SHA256 is something on the order of the power output of a trillion suns for a billion years (or some other meaninglessly-prohibitively-large amount). Despite such metrics and off-the-shelf solutions being readily attainable within this field Apple has implemented a system that has resulted in a collision within days, followed by a free automated collision generation tool.

This demonstrates absolute gross incompetence in design, and permanently eliminates any trust any sensible individual can ever have in the system despite your protestations.

There is simply nothing you can say to convince any sane person otherwise, the situation has been factually described and it is what it is.