> "Apple" wasn't scanning your phone, neither was there a "backdoor".

Yes, you're right. But I've seen calls for phones to scan all content before being uploaded or encrypted, and it often feels, at least in some countries, that could still plausible happen. I suppose that's what I had in mind when I wrote my comment.

> But the exact same risk exists when you upload stuff to the cloud anyway and on an even bigger scale.

There's a difference with them actively doing it and announcing they are doing it, vs the possibility they are doing it silently without consent.

ALL cloud providers are actively scanning all of your content right now, unless you specifically encrypt it yourself.

It's a cost of doing business pretty much, you need to give the authorites access to customers data or they can go "but think of the children, there might be child abuse material in there!" and that's really damn hard to argue against.

Thus: checking stuff on-device and keeping the cloud locked so that not even the hoster can access it nor can they create a backdoor.

> you need to give the authorites access to customers data

At least some cloud providers require a warrant before doing so.

Of course they need a warrant. Officer Johnson from Randomtown Alabama can't just call up Google and tell they want access to everything in GDrive :D

But the point is that if the data is fully encrypted, no warrant will help against pure mathematics. A cloud provider cannot give something they have no access to.

> Of course they need a warrant.

Right, but your post kind of made it seem like they didn't. And with the CSASM stuff, they didn't need anything to match hashes. So what happens when that gets expanded to other types of content that they still don't need a warrant for? Like torrent files?

> But the point is that if the data is fully encrypted, no warrant will help against pure mathematics. A cloud provider cannot give something they have no access to.

Agreed, but most don't use encryption, and should still have privacy rights for their data.

If they would have expanded it beyond CSAM, in that instance I would have joined the rest of the internet on the picket lines protesting against the system.

The point of the system would've been that EVERYONE'S content would've been encrypted in the cloud without them needing to do anything.

i thought they can't scan the media in icloud, since media is encrypted, no?

also: If it was someone sending you hashbombs of intentional false matches or an innocuous pic that matched because some mathematical anomaly, the actual human would notice this instantly and no action would've been taken. - if someone is doing this, imagine the scale- thousands of pics that should be human-evaluated, scaled to thousands of people, it'll be just plain ignored, meaning system loses it's purpose. also, you say that it'll be enabled only if icloud bck is enabled, but it's not guaranteed, this assumption can later change... and it doesn't make sense, for me your 2 statements contradict themselves:

- if apple can scan your photos in icloud AND for this feature to be enabled, you must enable icloud, why they should send hashes to you? they can scan the photos anyway in icloud, since all your photos are backed up there. Unless... they can't scan photos in icloud since these are encrypted, meaning scanning can be done only locally before photos are sent, meaning icloud enabling is not mandatory and it could work without it.

Either way the csam scanning is imo pointless, on one hand bc of privacy reasons(and we've seen that if state is able to use a backdoor, it'll use it when needed) and on the other hand, because of generative algorithms: photos can be manipulated to trigger csam even if human eye can see another thing (aka hashbomb) OR a sick/ill intentioned person can generate a legit csam like photo just by using target ppl's face(or description of their face), in this case I don't even know if they are breaking the law or not, since the image is totally generated but is looking totally illegal

You do know that we currently have "thousands of people" watching for and tagging the most heinous shit people upload to social media, right? There are multiple sources for this how we use outsourced people from Africa and Asia to weed through all of the filth people upload on FB alone.

"Looking illegal" isn't enough to trigger a CSAM check in this case. It's perfectly normal to take pictures of your own kids without clothes in most of Europe for example. Nothing illegal.

That's why the checks would've been explicitly done on known and confirmed CSAM images. It wasn't some kind of check_if_penis() -algorigthm, or one of the shitty ones that trigger if there's too much (white) skin colour in an image.

Again, somebody can train an algorithm to create false positives or real csam like pictures, like close enough to trigger the check. Afaik csam is not about exact match but rather close enough match based on a clever hashing algorithm, and in this case, algorithm can be induced into false positives(and by my limitet knowledge, hashing can have collisions) or even true positives but that are fully generated(and afaik generated images is not illegal, but i guess it depends on country).

Outsourcing work for this(afaik) isn't possible since it's private data, not public and only specific organisations can have full access to potential triggers

But in the end it also doesn't matter because there are other problems too, like how to make the final list easily checkable so that we are sure governments/ companies do not alter the list to target specific ppl/groups for their own interest. Or how algorithm isn't modified under the hood to check not just images but also text/files

People DID get into a huge fuss and started building false positive generators in a huge wave. Like they were proving something or pwning apple.

Nobody read the bit about an actual human verifying results before any law enforcement would be called in.

And outsourcing checking is a huge industry even today[0]. How do you think the huge social media companies keep CSAM, gore etc out of their systems? They're not using Pied Piper's hotdog or not algorithm, that's for sure.

[0] https://www.theverge.com/2019/2/25/18229714/cognizant-facebo...

A snippet from the article:

> The video depicts a man being murdered. Someone is stabbing him, dozens of times, while he screams and begs for his life. Chloe’s job is to tell the room whether this post should be removed.

And outsourcing checking is a huge industry even today[0]. How do you think the huge social media companies keep CSAM, gore etc out of their systems? They're not using Pied Piper's hotdog or not algorithm, that's for sure. - is this done for private data or public? Public data check can be outsourced no prob, not sure abt private.

Nobody read the bit about an actual human verifying results before any law enforcement would be called in. - again, if system can be gamed, human check is useless. Imagine someone will generate 100k false positives, multiply this by 1k ppl and imagine among them there is a real bad person with 10 real csam images and another person with fully generated csam images (like generated porn is a thing now so generating csam like images is possible). How do you think gov will human evaluate 100kk pictures that are triggering the system? Because if they can't, system is useless for this usecase but is still useful for potential gov oppression or company ad targeting

You can imagine all you want, but you're still wrong.

You can generate a billion false positives and it still won't do anything. You need to get them to people's iCloud photo libraries first. Each library needs to have multiple false positive images before triggering a human check. They intentionally didn't tell the exact number needed, but it's not two.

If you have a way to get enough people to get fake CSAM material on their phones to overwhelm the human checkers, why would you waste in on something stupid like that? Just by forcibly inserting advertisements to peoples Photo Libraries would make you a billionaire. Not an ethical one, but still rich.

Oh, and just for reference. FB gets 350 million photos uploaded every day and they keep it moderated just fine. The amount of people you'd need colluding with you to overwhelm the system Apple had designed would be staggering.

And then you've achieved what? Make it possible to share child pornography because you broke the system? Yay, victory?

You might've read the spec but you're missing the point and your approach is naive. For me it's about crossing the line. If you want to be snooping around my phone or my house, you need a warrant and go to official channels provides by my gov officials. And you really think it's as simple as picking apples from oranges? I mean come on. Yes, it's easy to implement hash check to see if u have some known child porn in your cloud. But was that the use case for the advocates? No. Their use case was to try finding abuse and that would need a more thorough scanning. And once we're there, we have to make hard decisions on what is porn or abuse. If u think it's easy, then you need to it through harder. Think of some picture from sauna where there are naked family, might be harmful? But normal here in Finland. What about a stick figure cartoon what depicts a some shady sexual positions with a smaller child-like figure in it? Or what about grooming, asking for naked pics? How is this system going to prevent that? I mean, I get why ppl would want something like this. But it isn't the right solution imho.

First of all, your analogy of "sending officials to your home without a warrant" is again you completely misunderstanding the feature.

The local scanning would've been enabled only if you would've uploaded the photos to iCloud anyway.

To complete your crappy analogy: You had already agreed to send the information of everything contained in your apartment to a company, and now you're getting into a hissy-fit when they are sending a robot visit your house with a list of known child pornography photos and checking if you have any?

There weren't any "hard decisions" it wasn't a "this has a naked person in it" -scanner. It wasn't an "abuse checked" it was specifically created to find known CP.

It checked your photos against a NeuralHash of KNOWN child pornography. So the only chance your sauna photos would've been flagged if they were included in a set of CP photos distributed widely enough so that they're added to the CSAM database.

And nobody claimed the system would magically cure the world of grooming, where did you get that idea from? Although the current filters in iMessage might help if the abuser is stupid enough to use that for grooming.

Sorry for my previous posts' hostile nature. Thank you for clearing up the actual proposed implementation details. It sounds like I came to pretty much the same conclusion as you did. I might have been quite unclear, but I was actually referring to (in my mind) the inevitable stage 2 of the implementation, if you give the little finger here ... theres no stopping. When I read about this, I was feeling sad, frustrated and a bit angry too. I like my iphone, i switched from android because they aged too quickly. Given the nature of Apple ecosystem, I took this as crossing the line. And i would have to strongly look for alternatives, since this scanning is not something i want to accept and i would vote with my money elsewhere.

Yes, i know they scan all photos already. But to scan MY photos and hold me accountable, that is something i am not okay with. Yes, its only icloud, for now, but i dont see the point of using some 3rd party cloud, i used to, but in apple device it would be suboptimal for my daily use etc etc.

My point was of the original case of protecting children. That is what the advocate groups want. That is what I want. But I just draw the line there what i feel is my personal space, that you don't scan my photos or files, and that is not the answer. Since there are these few behemoth FAANG companies ruling the ecosystems and impacting our lives, the battle of privacy needs to be fought right there. So, for me that would be the line as I see the progression of that path is much worse. That is where I raise my hand and say, I'm out, what next? No smartphone? Maybe. :) Peace.