> Under no circumstances is it justifiable to essentially enlist people's devices to police their owners, while using the electricity that you pay for, the internet service you pay for, and the device itself that you pay for to perform a function that is to absolutely no benefit to the user and in fact can only ever be harmful to them.

This is an obvious misrepresentation. An opt-in system to detect when adults are trying to groom kids by sending them porn seems like the opposite of harm. I imagine a lot of parents want that.

As for scanning what gets sent to iCloud. That’s also an opt-in service, and frankly it seems entirely reasonable for Apple not to want their servers to be used as a repository or hub for child porn.

It's not a misrepresentation whatsoever.

If Apple wishes to scan what's on their servers, that is their prerogative. They can use their compute resources and energy to do so. You needn't install spyware on a person's device that is of no benefit to the user. I'll reiterate, this can only ever be harmful to the user. Its utility right now is at its absolute best and most altruistic and it is still a violation of people's privacy and stealing computing resources from the device owner.

> That’s also an opt-in service, and frankly it seems entirely reasonable for Apple not to want their servers to be used as a repository or hub for child porn.

This will not stop their servers being used as a repository for illicit materials, if that is what you're suggesting.

Maybe they want to turn iCloud fully e2e, so cannot actually scan anymore on their servers.

They have to scan for CSAM by US law.

Could someone point to the relevant statute/regulation?

As far as I'm aware, there is none, though the EARN IT act that passed the Senate last year would have brought us closer by opening a company to liability if they provide E2E encryption without a mechanism for CSAM detection. https://en.wikipedia.org/wiki/EARN_IT_Act_of_2020

They don’t have to scan by US law. They only have to report detections, so if you have a content moderation team, they must report.

The EU and UK are in the process of passing laws requiring scanning.

> stealing computing resources from the device owner.

No, it is opt-in. Nothing is being stolen.

> "As mentioned in the article, this does absolutely nothing towards protecting children other than directing all but the biggest idiots towards platforms that can't be linked to them, which I'd imagine, they already are."

I suspect you're more wrong than you think about this. People share large volumes of CSAM through lots of different services - I knew someone who worked on the problem at Linked In(!).

HN likes to downplay the actual reality as if it's always some trojan horse, but the issue is real. It's worth talking to people that work on combatting it if you get the chance. I'm not really commenting on Apple's approach here (which I haven't thought enough about), but I know enough that an immediate dismissal based on it 'not helping' is not really appreciating the real tradeoffs you're making.

You can be against this kind of thing from Apple, but as a result more CSAM will be undetected. Maybe that's the proper tradeoff, but we shouldn't pretend it's not a tradeoff at all.

"Robin Hanson proposed stores where banned products could be sold. There are a number of excellent arguments for such a policy—an inherent right of individual liberty, the career incentive of bureaucrats to prohibit everything, legislators being just as biased as individuals. But even so (I replied), some poor, honest, not overwhelmingly educated mother of five children is going to go into these stores and buy a “Dr. Snakeoil’s Sulfuric Acid Drink” for her arthritis and die, leaving her orphans to weep on national television" [0]

[0]: https://www.lesswrong.com/posts/PeSzc9JTBxhaYRp9b/policy-deb...

Edit: After digging in, HN commentary is missing the most relevant details about this particular implementation. iCloud image checking compares to known CSAM image hashes - this means effectively zero false positive rate.

For iMessage child account ML scanning it’s on device and just generic sexual material restriction - more similar to standard parental controls than anything else (alerts only go to the parent, and only happen on child accounts)

My initial impression is this is a good approach. The risk mostly comes from future applications (like the CCP adding hashes of other stuff that isn’t CSAM like tankman or something).

The frankly ignorant knee-jerk responses from technical HN readers do a disservice and weaken the ability of technical people to push back when necessary.

One of the prerequisites to "discuss the tradeoffs" of this ordeal is the trust that the people behind it, and those in the position to push things further are actually interested in keeping the scope limited.

There are diminishing returns to using the same old excuse again and again. I'd say most people are just tired of the whole "Just think of the children!" into "Ah we got this system in place why not use it against <a little less evil but still illegal thing> too" followed by "It's the law in China/Turkey/Russia/wherever, <Company> can't just ignore it (and thus not help putting reporters, critics and other people into prison)" combo.

What you are saying is basically another rendition of "look the problem of child abuse exists and this could help so it is worth discussing", which is also a variant of the same.

> iCloud image checking compares to known CSAM image hashes - this means effectively zero false positive rate.

Actually we recently had news where an Apple tried to help cover up their errors [1], the system was supposed to be safe™, doesn't mean the people having control over it can't make mistakes, or worse.

What details are being missed here, exactly? Ultimately it is trivial to expand the hashes to compare to, isn't it? What does it matter that they use CSAM for now? It doesn't remove the involvement of humans in controlling the system, so false positive rate will never be "effectively zero", and it can easily be expanded.

We are left with the same two arguments as always:

- Think about the children

- Just trust the company, they use technology™, no you aren't allowed to check. Yeah they can easily abuse it, but we don't know for sure!

I wouldn't say HN is ignoring details, many people are just tired of the same old loop, there is no reason to put trust into these endeavors and it is reasonable to doubt even the motives.

[1]: https://news.ycombinator.com/item?id=27878333

It does not imply a false zero positive rate at all, because the hashes used are by necessity fuzzy, and even a completely benign picture can trigger a match.

Now they're saying that if the match is a false positive, it'll be screened by the human reviewer. But that means there's some stranger there potentially looking at my private photos (and, by the way, I wonder which country they'll be located in?). That's already completely and utterly unacceptable - you don't have to wait for any "future complications".

You need to stop assuming everyone is commenting out of ignorance and read up on how perceptual hashing works. It’s about as far as you can get from zero false positives, if configured otherwise it becomes just a poor version of SHA or whatever and can only detect exact matches.

At the time of my comments people weren’t discussing the fuzziness of perceptual hashing - they weren’t discussing implementation details at all.

I don’t know enough about the specific implementation or perceptual hashing details and probably pushed back too hard as a result of the other comments at the time (which were comments out of ignorance).

The level of downvotes I received is disproportionate to what I wrote anyway - this is clearly a political issue on HN. The irony is I’d probably align more with the risks being too high position, but it needs to be considered after actually understanding what the true risks are first.

Apple is obviously already scanning for CSAM for whatever non-E2E content hits their servers. With this, they offload some of the cost of computation to the end nodes but that can’t be the only reason for pushing this tech, so it’s only natural to ask why and look for avenues of exploitation. The Trojan horse/foot-in-the-door hypothesis is basically Occam’s razor.

This Daring Fireball blog post is a better summary than the discussion I’ve found on HN: https://daringfireball.net/2021/08/apple_child_safety_initia...

Occam’s razor is that they’re doing what they say they’re doing and not some more complicated future action.

How will you know? And actually, how will Apple know that's all their doing? They won't because all they can see is a hash, so they don't actually know if they're checking for anti-winnie the pooh memes, or CP. And there is no public auditing, there is no auditing by your senators, there is no public disclosure of the hashes they are checking, and no public disclosure of the exact algorithm used. And even if they tried to do the latter two, you'd have no way of verifying it.

You said earlier upthread, that HN is ignoring the tradeoffs. I find that distasteful, people have considered the tradeoffs of this and they don't like them, but that doesn't mean they need to preface every single statement about the thing with "well actually i've considered the tradeoffs, and I've found them to be blah, and..." That would quickly get tiresome which is why people don't do it.

I'll make a different, similar generalization. Like most people pushing this, you are pretending to care about the "tradeoffs", but you don't. You don't care about the downsides, you just say you do, and then blithely ignore them when they are brought up. What's the golden rule again?

I would say most (if not all) know how easily you can change the scope of a database like this(like you said yourself) to check for other inconvenient terms, links, memes, etc that do not "toe the party line"(whether is eastern or western - I don't care) and shut down inconvenient discourse.

The point is: This is too easy to abuse.

Being specific with the arguments and addressing the nuance matters imo.

Most of the HN responses are dumb, get the details wrong, and don’t take the trade offs seriously. That kind of thing causes me to dismiss them entirely.

There are legitimate arguments and risks which I’d concede, but they’re not what most people in the comments are talking about.

The trade-off here is to get pedophiles off of Apple services and on to something else, we give Apple an entry point into examining our private data. I understand it's hashes now, that does nothing to prevent this from expanding.

"People just don't understand!!" has never been a convincing argument.

This is just a slippery slope argument.

The implementation specifically for detecting CSAM can be okay while using that for other purposes can not be okay.

The goal is to stop child sexual abuse, FB reports millions of cases a year with a similar hash matching model for messenger.

> ""People just don't understand!!" has never been a convincing argument."

That's not my argument - I just think most of the HN comments on this issue both miss the relevant details, and are wrong.

Slippery slope fallacy is often implied incorrectly when it comes to people. Human nature is subject to the "foot in the door" sales tactic.

https://en.wikipedia.org/wiki/Foot-in-the-door_technique

I think HN takes the incorrectly perceived moral high ground and ignore the fact that there is a real duty to act here.

There are people who profit from CSAM and in turn this creates a market for abuse. Unless we create structures which disincentivizes these behaviors — right now there are none - they will continue to grow. What Apple is building will basically make any criminal who sells to someone sloppy enough to store in iCloud risk being traced as the origin of the CSAM. Back to the darkest web they go.

Anti CSAM is inevitable. You will find similar systems for all major providers eventually.

I'm pretty sure child abuse is a criminal offense everywhere. That's quite a disincentive.

Is CSAM a specific list of images that does not change, or does a government body actively control this list? Is there anything that would technically prevent the addition of a specific image of an enemy of the state? Hypothetical question, how will Apple respond when the FBI asks Apple to scan for images they know are also on the phone of the latest domestic terrorist? And how likely do you think the FBI will try to expand the scope of images that are scanned?

I don't know - I think these are the good questions and the things I would want to know in more detail. I think this kind of thing is where the risk lies.

I think this is why getting the details right matter because if people are arguing about unrelated nonsense it's harder to focus on the actual risks represented by the questions you're asking.

> You can be against this kind of thing from Apple, but as a result more CSAM will be undetected.

You cannot claim to be making "the real reasonable analysis" and write this. So much for "you're all geeks stuck on technical details". Quite the contrary: I'm sick of bogus software pretending to solve problems for me, while the quality of tech has exponential degraded over the last 20 years (often due to trying to solve some unsolvable problem in a bad way that backfires).

Now imagine you have a 16 year old girlfriend. She sends you a nude photo. Your phone calls the cops on you (it doesn't matter if the phone doesn't quite do this now, it will in the future. They will use their ML crap to detect the age of subjects in photos and explicitness of the photo). You normally wouldn't go to jail for this since 16 is legal in 99% of the civilized world, but thanks to America with their super duper "non-technical" innovations that only big boy white collars can understand, you can go to jail for having a photo of your legal girlfriend.

Mostly good...

> big boy white

but why the totally uncalled racism and sexism here?

It does nothing to strengthen your argument and it is just dumb.

I am truly sorry for your loss in that your brain is implemented using regex.

I meant "white collar big boys", but I did not bother to edit as I'm writing.

The guy above is claiming everyone who is against apple's yet-another-bogus-TPM-style-snakeoil is a little geek who does not understand anything outside their little tunnel.

Also now that I re-read his comment:

> Edit: After digging in, HN commentary is missing the most relevant details about this particular implementation. iCloud image checking compares to known CSAM image hashes - this means effectively zero false positive rate.

False: it's a perceptual hash. Ignoring the fact that if for some reason you choose to let people host stuff in your icloud account (perhaps as a neat hack), which may be out of terms of service, but certainly not worth 20 years of jail: perceptual hashes have false positives, and can confuse images that appear harmless but were crafted to look like $badimg. But you don't have to be technical to understand that having your devices police you is bad, you just have to not be blinded by politics and boogeyman your state has sold you.

It’s not just HN folks, go read the “open letter”.