There's a simple answer to this right? Despite everyone's reaction, Apple genuinely believe this is a novel and unique method to catch CSAM without invading people's privacy. And if you look at it from Apple's point of view that's correct: other major cloud providers catch CSAM content on their platform by inspecting every file uploaded, i.e. total invasion of privacy. Apple found a way to preserve that privacy but still catch the bad people doing very bad things to children.
> other major cloud providers catch CSAM content on their platform by inspecting every file uploaded, i.e. total invasion of privacy.
> Apple found a way to preserve that privacy ...
So scanning for CSAM in a third-party cloud is "total invasion of privacy", while scanning your own personal device is "preserving privacy"?
The third-party clouds can only scan what one explicitly chooses to share with the third-party, while on-device scanning is a short slippery slope away from have its scope significantly expanded (to include non-shared and non-CSAM content).
While local, on-device content access has no such guarantees - it's a question of policy rather than technical infeasibility.
And policies change.
> They are scanning files that are being uploaded. So, yes.
Even considering the situation as it is today, the cloud providers (as well as Apple) are scanning just the files that are being uploaded. So I'm still not sure how one is "total invasion of privacy" and the other "preserving privacy".
No, the OS can limit access to local content. E.g. Mac has https://support.apple.com/guide/mac-help/control-access-to-f..., and ios has its own permissions system.
It also requires installing third-party software on the device. While if Apple's policy regarding local content changes in the future, there is no way to avoid the OS itself (other than jailbreaking I suppose).
> Apple is trying to avoid scanning in the cloud so that the data can be E2EE
Citation needed.
> is a short slippery slope away
Obviously people who trust Apple aren't concerned about slippery slopes. What's the point of your post?
There’s this bizarre notion that using end-to-end encryption can absolve you of responsibility, that the authorities will have to accept an answer of “we literally can’t access it”.
That’s just not the case for centralised things: you’re deliberately facilitating some service, government will find you liable for some things in its operation, and if you don’t comply, they’ll fine or shut you down. E2EE doesn’t absolve you from law; law is all about saying you’re not allowed to do things that are physically possible.
(Decentralised things, now they can be banned but not truly stopped because there’s no central party to shut down.)
A state that does not accept it might retaliate against the entity giving that answer or forbid future use of end-to-end encryption without backdoors, but the truth of the answer doesn't depend on anyone's acceptance.
This is what has happened in many countries already.
Legal terms such as "murder", "fraud" and "rape" do change as effect of regulatory changes. "Encryption" and "privacy" do not.
There's a limit to how much you can bend semantics in your PR before it breaks and you get backlash.
They're deliberately misrepresenting what's happening, appearing surprised when people misunderstand, and bundling together legitimate criticism with misunderstandings.
I can draw some parallels to how Google went out with FLoC.
Honestly I can't tell where Hanlon's razor should cut here.
Problem is that the law is self-contradictory and it is up to the judicative institutions to fix it as soon as possible.
You can still do secure backups of your phone without using iCloud, but there isn’t a way for Apple to do end to end encryption of backups transparently like you can with real time communication. The only way end to end encryption of backups works is to require people keep a separate secure key(s) to avoid losing their data, which means a universal implementation has real direct risk for users.
As long as Apple has access to these files the FBI can legally require them to do these searches. From a pure PR perspective they should have communicated what was already going on before releasing this system because people assume something significant changed.
There is no reason the password can't be the encryption key, with backup keys stored with a trusted third party (eg: your credit union or bank) without notation as to what these backup keys are tied to.
Trusting third parties with the password in unencrypted form is either systematic in which case the FBI now just needs collect data from 2 different organizations, or on a case by case basis in which case users will mess it up. Apple etc would have no way to verify users actually did something to back up their keys.
Apple’s current approach is to let users setup their own backups if they want security which allows for privacy just fine without providing a service with fundamental issues.
Apple has already proven it will concede to China's demands.
They are building the worlds most pervasive surveillance system and when the worlds governments come knocking to use it ... they will throw their hands up and feed you the "Apple complies with all local laws etc.."
They control what goes into the on-device database that is used.
>The on-device encrypted child abuse database contains only data independently submitted by two or more child safety organizations, located in separate jurisdictions, and thus not under the control of the same government
https://www.techwarrant.com/apple-will-only-scan-abuse-image...
It's bizarre to me that people are freaking out about governments adding client side hashes but no concern that they could be doing server side checks.
I tried disabling iCloud keychain and it just flips back on. Sometimes it asks for a login first. Sometimes it shows a cancel/continue modal. Either way, it magically flips back on. No error message.
I tried backing up my device to my hard drive (with Photos already on iCloud) and it kept complaining that there wasn’t enough space. It throws error message after warning message that your content will be deleted. It created additional copies of my photos each time my phone synced.
To properly back up, I had to copy the photos directory to an external hard drive, delete the original, mark the external hard drive one as the system one and then finally free up enough space to back up. iCloud and the device backup weren’t smart enough to free up space for my backup. In fact, I first backed up all my photos to iCloud first because they said that it would free up space on my hard drive as necessary. LOL.
BTW, iCloud keychain is still on for me. Fuck Apple.
I think I'm not the only one who'd rather not have my devices call the cops on me in a country where the cops are already way too violent.
Further, this is step 1 of a process they have explicitly said they are looking to expand on [1], even going as far to state it in bold font with a standout color.
So theres no telling that they wont expand it by simply scanning everything, regardless of icloud usage, or pivot it to other combat "domestic terrorism" or "gun violence epidemics" or whatever else they feel like.
Its an erosion of trust, even if not a full stop erosion, its something they intend to expand upon and wont be taking back.
[1] https://www.apple.com/child-safety/pdf/Expanded_Protections_...
This is the key point.
1. What if I change my mind and decide not to upload the picture?
2. This is a new mechanism for scanning private pictures on the device. What could go wrong?
> If we can't trust Apple to follow their promise, their products should already have been considered compromised before this change was announced.
Many people did trust Apple to keep their files private until now.
No it isn’t. It’s a mechanism for scanning pictures as they are uploaded to iCloud Photo Library.
Private pictures on the device are not scanned.
Also, what apps are you talking about?
As for apps - WhatsApp, for example, saves everything to Camera Roll by default. Which then gets auto-uploaded to iCloud.
iCloud backups are not scanned. Only photos uploaded to iCloud Photo Library.
> As for apps - WhatsApp, for example, saves everything to Camera Roll by default. Which then gets auto-uploaded to iCloud.
This is not true at all. The user is asked for explicit permission before apps can save photos to the camera roll, and the user must choose to use iCloud Photo Library.
>Many people did trust Apple to keep their files private until now.
And that was my original point. If a pinky promise from Apple is not enough to trust them, then Apple should have never been trusted.
You can choose to upload many pictures. They will start uploading. Then, you change your mind. Some pictures were not uploaded yet. But they were scanned by the new algorithm.
That wouldn't be a problem in an ideal world, but the one in which we live is far from even resembling one. Mining data is already a huge business, and governments everywhere would love tools to use to get advantage over people they don't like. There's huge motivation and demand for those tools at all levels, and at least governments have the resources to buy them and the power to force whoever implements them to stay silent. I'm not implying that spyware tools exist in any phone, PC, smart TV, car, etc. because we can't prove they don't; that's the argument used for UFOs, witches and unicorns, no thanks, but we better think like they do because technology, resources and demand for their adoption are real, and the rest is probability.
Apple are not uploading anything to government organisations, in fact they are uploading less than google, by their own(and googles admission).
You have 0 proof that they are uploading to gov orgs… right?
I don’t mind my one drive being scanned for “bad stuff”, I very much mind my personally owned data stores being scanned, with no opt out.
If you turn off iCloud Photos, nothing is scanned.
Microsoft scans everything.
>The system that scans cloud drives for illegal images was created by Microsoft and Dartmouth College and donated to NCMEC. The organization creates signatures of the worst known images of child pornography, approximately 16,000 files at present. These file signatures are given to service providers who then try to match them to user files in order to prevent further distribution of the images themselves, a Microsoft spokesperson told NBC News. (Microsoft implemented image-matching technology in its own services, such as Bing and SkyDrive.)
https://www.nbcnews.com/technolog/your-cloud-drive-really-pr...
According to Apple. and For Now. Patriot Act was only for terrorists. Apple makes concessions for China. Creating this technology, makes it very easy for China to go, "Look at all photos, always". If they only want to scan stuff on iCloud Photos, no worries, just implement on it on their end. This tech does not need to exist in that case.
> Microsoft scans everything.
Everything uploaded to the Cloud. The Cloud bit is fairly important here. That's me, willingly, putting information on their property. Perfectly acceptable and fine for them to ensure that it's nothing unethical.
However, someone else snooping through your drawers looking for something to pin on you is not private, nor the same as checking their own property.
By your definition of "everything", both Microsoft and Apple scan "everything". (Or Apple will, after this new system is rolled out.)
Nobody is arguing about the legality of CSAM itself. It's an issue that is not popular to discuss, and of course being on the wrong side of it results in near-universal, justifiable derision. Stopping its spread is absolutely the right thing to do.
So at the point that companies will always be held liable for storing it, they will have to put up countermeasures of some kind or find themselves sued out of existence. Server-side scanning is one method, and Apple's on-device scanning is another.
There are certainly ways that Apple can go too far with whatever it happens to come up with as its solution to stopping CSAM, but there is still seems to have been a line behind which nobody particularly cares how the detection is implemented and life continues as usual. If Apple had chosen not to cross that line, maybe many of the arguments being made here would never have been brought up at all.
What it takes to accept is the "nothing to hide" mentality: your files are safe to scan (locally) because they can't be known CSAM files. You have to trust the scanner. You allow the scanner to touch your sensitive files because you're not the bad guy, and you want the bad guy be caught (or at least forced off the platform).
And this is, to my mind, the part Apple wasn't very successful at communicating. The whole thing should have started with an educational campaign well ahead of time. The privacy advantage should have been explained again and again: "unlike every other vendor, we won't siphon your files unecrypted for checking; we do everything locally and are unable to compromise your sensitive bits". Getting one's files scanned should have become a badge of honor among the users.
But, for some reason, they tried to do it in a low-key and somehow hasty manner.
Everybody knows what goes on behind that door. And yet, everyone would much prefer to close it. Apple's method is equivalent to the toilet door being removed, so that you cannot do anything nefarious behind that door.
Your phone also (on average) have nothing to hide, but that's also why you need privacy on your devices.
That's purely specious, and you say it as if I ever asserted there aren't "dumb people of every type.", which I didn't say.
Yes, there are dumb people of every "type". Are there as many intelligent people as those of low intelligence who struggle to think abstractly, struggle to read and write, have poor motor control, rely heavily on dogma for a moral compass, or have trouble with impulse control?
If you think there are as many people with a high IQ as a low IQ with those traits, I don't think you're being honest.
Here's some data for you:
https://www.ncbi.nlm.nih.gov/pmc/articles/PMC4478390/
https://pubmed.ncbi.nlm.nih.gov/14744183/
https://pubmed.ncbi.nlm.nih.gov/961457/
https://www.dw.com/en/scientists-find-brain-differences-in-p...
https://sci-hubtw.hkvisa.net/10.1023/a:1018754004962
https://www.livescience.com/4671-study-pedophiles-tend-short...
https://pubmed.ncbi.nlm.nih.gov/29625377/
Are most pedophiles of low intelligence overall? It's doubtful, but there's room for more study.
Are the pedophiles that are offenders of low intelligence? Significantly more offending pedophiles are of low intelligence than the general population. Yes, the factors at play may also apply to other forms of crime. But when you factor out non-offending pedophiles, those we aren't as concerned with in regards to CSAM, the ones that are caught aren't exactly the cream of the crop.
Furthermore, I've met a handful of pedophiles, both convicted and not. Every one of them either was clearly mentally retarded or bordering on retardedness, or they had a severe personality disorder. Yes, it's a small sample size, but the ones that were caught did so doing the dumbest of shit (putting aside the horrific nature of child exploitation in the first place). This even includes a female pedophile, who was both stupid and psychopathic, and I find it funny that one of those studies questions the very existence of female pedophiles.
There are many pedophiles that are of above-average intelligence. Most may not even be considered "dumb". But to imply that there's no difference in distribution of, frankly, dumb people to smart people between different behaviors is absurd in that it ignores any sort of relation implication that intelligence has on said behaviors.
As a group, the data does not suggest that pedophiles are exceptionally intelligent, but rather the opposite.
So yes, while I believe there is a privacy concern (I said earlier I would not want such software or hardware running on my own compuer), I stand by my conclusion that CSAM detection would catch a significant number of pedophiles and probably continue to do so for some time. The point of such systems isn't to be a superweapon to catch every pedophile. The ones that it might catch happen to be the ones who are most likely to offend. If you play the stupid game of uploading CSAM to Apple iCloud, you win the stupid prize of getting convicted, and it just so happens that a ton of pedophiles are likely to do something that boneheaded.
I’ve reviewed much of the existing data on minor-attracted persons (which is scant). The problem is that the samples are heavily biased towards including individuals who are caught or self-admit for care, which is not a very good representative sample. As a few of the studies and articles you posted mention, there was little study done until very recently. You also make the leap without supporting data that this system would catch those most likely to offend. Generally I would say the data is insufficient to conclude in any direction other than dumber people get caught doing crimes at a higher rate than smarter people.
Lastly, I have some serious doubts about your expertise in mental health and the study thereof from your use of “mentally retarded” and “retardedness”. You should know for future discussions that the clinical terminology has shifted to “intellectually disability”.
Maybe that's true of the ones that get caught, but if so there's a good chance that at least part of that is inverse survivorship bias. (Similar to “Why are most of the CIA covert missions you hear about failures?”)
You may catch a few perverts looking at the stuff, but I'm not convinced this will lead to catching the producers. How would that happen?
Compare this to the war on drugs. There is huge demand for drugs. Many drugs are highly illegal to possess, and a lot of people get jailed for possessing them. And yet most of us consider the war on drugs to be an abject failure that hasn't done much more than create inequity.
Why should something like CSAM possession be any different? I wish it was different, and these sorts of tactics would reduce this awful problem, but I'm just not convinced it does.
What’s novel about this? The technique and issues with it are fairly obvious to anyone with experience in computer vision. It seems not too different from research from 1993 https://proceedings.neurips.cc/paper/1993/file/288cc0ff02287...
The issues are also well known and encompass the whole subfield of adversarial examples.
Saddened by the privacy-adverse functionality on handsets but Apple seems to be hitting the nail on the head that poor communication principally is driving outrage.
So it's not even accomplishing "keeping it off the servers."
Apple's actions will be effective at making sure every CSAM aficionado will not trust their device. Or, if they're tech savvy, they will at least know not to co-mingle their deepest darkest secrets alongside photos of their mum and last night's dinner.
If you think Apple's very big, very public blow-up is making waves in the hacker/security/libertarian crowd, just imagine how big it's blowing up in the CSAM community right now. I dare say it's probably all they've been talking about for the past two weeks. Unlike places like Hacker News where there's plenty of people desperate to drag Apple through the coals, the CSAM community will be highly motivated to have a precise understanding of what Apple is doing. I dare say that they'll be extremely well informed about exactly what Apple's plans entail and how to work around them.
This is Orwellian doublespeak.
You don't preserver someone's privacy by snooping on their devices.
That is very unlikely. Most likely they compare some hash against a database - just like apple.
Eventually it could lead to Apple platform not having any CSAM content, or any known legally objectionable content, because any sane perpetrator would migrate to other platforms, and less sane, caught.
This, of course, takes the user base to overwhelmingly agree that keeping legally objectionable content is a bad thing and should be exposed to police, and to trust the whatever body which defines the hashes of objectionable content. I'm afraid this order is not as tall as we could imagine.
And yet photos that get scanned are still uploaded to iCloud Photos, so they do end up on Apple's servers.
Doing so right while activating a new iDevice is the way to prevent its private keys from ending up in iCloud, and so preventing Apple, or law enforcement, or some malicious hackers from cracking into your device.
I'm failing to see the issue.
The results of the scan looking for kiddie porn cannot be read by Apple until the device finds 30 examples of photos that match known kiddie porn, whereupon Apple gets the decryption key so they can see which images on their server need review, and a human review is triggered to make sure there haven't been 30 false positives.
I implore you to read the comment I originally replied in order to understand the context of my replies. Personally, I don't care what you're shocked about or not, as my OP wasn't directed at you at all.
After that, they get the decryption key and trigger a human review to make sure there haven't been 30 false positives at once.
That is better, and more private, in a very meaningful way.
False positive scan data sitting on the server is open to malicious misuse by anyone who can get a warrant.
>Innocent man, 23, sues Arizona police for $1.5million after being arrested for murder and jailed for six days when Google's GPS tracker wrongly placed him at the scene of the 2018 crime
https://www.dailymail.co.uk/news/article-7897319/Police-arre...
Also, I sincerely doubt that Google has ended it's practice of refusing to hire a human being to check for false positives when a poorly performing algorithm is cheaper.
I'm fine with the idea that if I upload stuff to someone else's server, they may take a look at it and maybe even punish me for what I've uploaded. Certainly if I encrypt the data before I upload it, they can't do that. But if I don't, then it's fine with me if they do.
But my device should not be snitching on me. Yes, this device-side scanning is supposedly gated on enabling uploads to iCloud, but that doesn't really make for much of a distinction to me. And since Apple certainly has the capability, they are likely one secret court order away from being required to scan even photos that aren't being uploaded, at least on some targeted subset of devices.
Apple has the ability to upload literally any software to iPhones, so this argument applies equally to literally any conceivable bad thing that software could do on iPhones.
1. The TSA agent opens your luggage and searches everything for banned items.
2. The TSA agent hands you a scanner for you to wave over your luggage in private, it prints out a receipt of banned items it saw, and you present that receipt to the agent.
Which one is more invasive?
3. The home builder installs a TSA scanner in all newly built homes. The scanners will scan all items as they are put away into drawers and cupboards, attempting to detect the presence of banned items.
Then the TSA Agents say they won't report you until at least 30 banned items are detected, even though you haven't flown in 10 years and don't have banned materials. As the TSA Agents walk back to their car, you overhear words like warrant, trouble, and tap amidst their chuckles. What could go wrong with that?
Which one is more invasive?
I don't think there is much to not understand. Apple is proposing putting a cop in your house. On device scanning is vastly vastly different than server side scanning. One is over apples property, the other is over YOUR property. Its a big deal, and a huge change in policy. And as myself and others have indicated from years of observation, it will grow and grow...
The scanner technology can also be used for detecting dissident journalism, but they say it won't be, it is just for preventing terrorism, but still they say, they really need to install it in your house even though it is only for scanning when you intend on going to the airport.
This has always been possible with server side scanning. Yeah we have to trust Apple to not enable it for users who don’t use iCloud photos but we’ve always had to trust Apple given their closed source system.
So does Apple.
EDIT: some people don’t like that answer, but “inspecting” in this context clearly means “digitally inspecting” (Google does not physically look at every file) and Apple does this with files that are uploaded. They do it in device, but it’s still inspected. That’s the whole point of this controversy, that there’s not much difference to people WHERE apple inspects and on device is actually arguably worse. Your sentence does not in any way distinguish what Apple does from what others do.
They're not going to find predators, all they are going to find is people incompetent enough to have years old (otherwise how would it end up at NCMEC?) CSAM on their iPhones and dumb enough to have iCloud backup turned on. To catch actual predators they would need to run AI scanning on phones for all photos and risk a lot of false positives by parents taking photos of their children on a beach.
Plus a load of people who will inadvertently have stuff that "looks" like CSAM on their devices because some 4chan trolls will inevitably create colliding material and spread it in ads etc. so it ends up downloaded in browser caches.
All of this "let's use technology to counter pedos" is utter, utter crap that is not going to save one single child and will only serve to tear down our freedoms, one law at a time. Want to fight against pedos? Make photos of your airbnb, hotel and motel rooms to help identify abuse recording locations and timeframes, and teach children from an early age about their bodies, sexuality and consent so that they actually have the words to tell you that they are being molested.
Who do you think has 30 or more CSAM images on their phone?
But let's also be real about something else. Think of the Venn diagram of child abusers and people who share CSAM online. Those circles are not identical. Not everybody with CSAM is necessarily a child abuser. Worse, how many child abusers don't share CSAM online? Those can't ever be found by Apple-style invasions of privacy, so one has to wonder if we're being asked to give up significant privacy for a crime fighting strategy that may not even be all that effective.
The cynic in me is also wondering what fraction of people working at places like NCMEC are pedophiles. It'd be a rather convenient job for them. And after all, there's a long history of ostensibly child-serving institutions harboring the worst kind of offenders.
> a crime fighting strategy
Is it a crime fighting strategy? I thought it was just about not making their devices and services into aids for these crimes.
> The cynic in me is also wondering what fraction of people working at places like NCMEC are pedophiles.
Good question. What has that to do with this?
And how many children can really be helped in this way? Surely the majority of child abuse happens independently of the online sharing of CSAM.
And is that really worth the suspension of important civil rights? That is really what this discussion is about. Nobody is arguing against fighting child abuse, the question is about how, and about what means can be justified.
Does it? I see no reason to assume that.
> And is that really worth the suspension of important civil rights?
No civil rights are being suspended. It’s not about that at all.
CSAM scanning is sold as beneficial, while in reality it won't do shit while it opens dangerous precedence backdoors!
Why do you assume it’s extremely old?
Apple announces that it is going to start scanning iCloud Photos only, and that their system is set to ignore anything below a threshold of ~30 positives before triggering a human review, and people lose their minds.
This is the difference between putting CSAM on a sign in your front yard (maybe not quite front yard but I can't come up with quite the same physical equivalent to a cloud provider) and keeping it in a password protected vault in your basement. One of those things is protected in the U.S. by laws against unlawful search and seizure. Cloud and on your device are two very different things and consumers are right to be alarmed.
I'll say it again, if you are concerned with this privacy violation, sell your Apple stock and categorically refuse to purchase Apple devices. Also go to https://www.nospyphone.com/ and make your voice heard there.
I honestly can't find the uproar here. Google devices can face match photos offline... so they are applying a neural net (scanning) ON THE DEVICE! How is that not worse than what apple do?
The difference can also be seen from a customer service perspective. One is a feature that lets you sort according to which friends you were with. The other is a feature that puts you in jail. No thanks. Not gonna pay money for that.
Literally no difference.
If you have illegal stuff only on your phone neither google or Apple will be notified or notify anyone else.
Let's say the TSA were to install air-travel-contraband scanners in everyone's homes, but promise only to scan things that are being put into your luggage as you prepare to go to the airport. And let's say that this became a requirement if you want to board a plane.
That's what this feels like. I'm fine with Google scanning through everything in my GMail account, or everything I've uploaded to GDrive, or created in GDocs. That stuff is on their servers, unencrypted, and I explicitly put it there.
But I'm sure as hell not going to let Google install something on my laptop (or phone!) that lets them look at my stuff, even if they pinky-promise that they'll only scan stuff that I intend to upload.
First of all, you have to be able to read it to do the comparison that can increment the counter to 30. So regardless of whether it is or is not encrypted there, they're accessing the unencrypted plaintext to calculate the hash.
And yes, on my device is definitively more private than on someone else's server--just like in my bedside drawer is more private than in an office I rent in a co-working space.
This doesn't matter because Apple can read iCloud data, including iCloud Photos. They hold the encryption keys, and they hand over customers' data for about 150,000 users/accounts a year in response to requests from the government[1].
How do you think Google and Microsoft scan everything in your account? They all have the capability to read your cloud data.
What Apple cannot read are the results of your device scanning your iCloud Photos. Those results are encrypted and stay that way until your device finds 30 matches for known kiddie porn.
Once you pass the threshhold, Apple gets the decryption key and a human review is triggered to make sure there weren't just 30 false positives.
maybe you mean to say that apple says they won't read it until that threshold has been crossed.
The kind Apple has built. You should read the docs. This is literally how it works.
>Apple is unable to process individual vouchers; instead, all the properties of our system mean that it’s only once an account has accumulated a collection of vouchers associated with illegal, known CSAM images that we are able to learn anything about the user’s account.
Now, why to do it is because, as you said, this is something that will provide that detection capability while preserving user privacy.
https://techcrunch.com/2021/08/10/interview-apples-head-of-p...
Meanwhile, a single false positive from an on server scan is open to malicious use by anyone who can get a subpeona.
Just going to respond to every post on here with these absurd points? K apple guy.
When that scanning gets moved from the cloud to being on your device, a boundary is violated.
When that boundary is violated by a company who makes extreme privacy claims like saying that privacy is a "fundamental human right"[1], yes, people will "lose their minds" over it. This shouldn't be shocking at all.
You would have to have 30 false positives before Apple can see anything, which is unlikely, but the next step is still a human review, since it's not impossible.
If anything, you should be outraged that Google and Microsoft have been scanning much more of your data, and doing so in a much more intrusive way.
Apple only scans iCloud Photos and they do so in a way that they can't see the results until they can be reasonably sure it's not just a false positive.
If you think Apple's approach is the best you're allowed to think that. I disagree.
Apple can't decrypt the results of the scan until the ~30 image threshold is crossed and a human review is triggered.
Given Google's reluctance to hire humans when a poorly performing algorithm is cheaper, are they turning over every single false positive without a human review?
Apple isn't scanning that.
Google and Microsoft are.
They don't cross that line like Google and Microsoft do.
With Apple, nothing but files you upload to iCloud Photos get scanned.
Facebook, even more so, they are explicitly anti-privacy to the point of being insulting.
Microsoft will happily show you everything they may send when you install Windows, you can sometimes refuse, but not always. They are a bit less explicit than Google, but privacy is rarely on the menu.
As for Amazon, their cloud offers are mostly for businesses, different market, but still, for consumers, they don't really insist on privacy either.
So that if any of these company scan your pictures for child porn, it won't shock anyone, because we know it is what they do.
But Apple claims privacy as a core value, half of their ads are along the lines of "we are not like the others, we respect your privacy, everything on your device stays on your device, etc...", they announce every (often legitimate) privacy feature with great fanfare, etc... So much that people start to believe it. But with that, people realize that Apple is not so different from the others after all, and if they bought an overpriced device based on that promise, I understand why they are pissed off.
https://techcrunch.com/2014/08/06/why-the-gmail-scan-that-le...
You don't consider the contents of your email account or the files you mirror to a cloud drive to be your own private data?
I expect that my ISP tracks and stores my DNS resolutions (if I use their DNS) and has a good understanding of the websites I visit.
I expect that an app that I grant access to my contacts uploads as much data as it can to their servers.
I expect WhatsApp and similar apps to collect and upload meta data of my entire photo library such as GPS info the second I give them access.
Hence, I don’t give access. And hence, it’s a problem if there is no opt-out of local file scanning in the future.
>So if iCloud Photos is disabled, the system does not work, which is the public language in the FAQ. I just wanted to ask specifically, when you disable iCloud Photos, does this system continue to create hashes of your photos on device, or is it completely inactive at that point?
If users are not using iCloud Photos, NeuralHash will not run
https://techcrunch.com/2021/08/10/interview-apples-head-of-p...
https://protectingchildren.google/intl/en/
> CSAI Match is our proprietary technology, developed by the YouTube team, for combating child sexual abuse imagery (CSAI) in video content online. It was the first technology to use hash-matching to identify known violative videos and allows us to identify this type of violative content amid a high volume of non-violative video content. When a match of violative content is found, it is then flagged to partners to responsibly report in accordance to local laws and regulations. Through YouTube, we make CSAI Match available for free to NGOs and industry partners like Adobe, Reddit, and Tumblr, who use it to counter the spread of online child exploitation videos on their platforms as well.
> We devote significant resources—technology, people, and time—to detecting, deterring, removing, and reporting child sexual exploitation content and behavior. Since 2008, we’ve used “hashing” technology, which creates a unique digital ID for each known child sexual abuse image, to identify copies of images on our services that may exist elsewhere.
The Google page has a section later down that also says they use hashing of images.
When Google scans on server, a single false positive result can be abused by anyone who can get a warrant.
>Innocent man, 23, sues Arizona police for $1.5million after being arrested for murder and jailed for six days when Google's GPS tracker wrongly placed him at the scene of the 2018 crime
https://www.dailymail.co.uk/news/article-7897319/Police-arre...
Apple's method is more private.
Mainland China will probably be the first chip to fall. Can't imagine the Ministry of State Security not actively licking their lips, waiting for this functionality to arrive.
Even though NCMEC describes itself as "private", it was established by and has been heavily funded by the U.S. government.
From an archive of NCMEC's own history page, cited on Wikipedia (https://web.archive.org/web/20121029010231/http://www.missin...):
> In 1984, the U.S. Congress passed the Missing Children’s Assistance Act which established a National Resource Center and Clearinghouse on Missing and Exploited Children. The National Center for Missing & Exploited Children was designated to fulfill this role.
> On June 13, 1984, the National Center for Missing & Exploited Children was opened by President Ronald Reagan in a White House Ceremony. The national 24-hour toll-free missing children’s hotline 1-800-THE-LOST opened as well.
$40 million/year of U.S. government funding from a 2013 bill (https://en.wikipedia.org/wiki/Missing_Children%27s_Assistanc...):
> The Missing Children's Assistance Reauthorization Act of 2013 (H.R. 3092) is a bill that was introduced into the United States House of Representatives during the 113th United States Congress. The Missing Children's Assistance Reauthorization Act of 2013 reauthorizes the Missing Children's Assistance Act and authorizes $40 million a year to fund the National Center for Missing and Exploited Children.
> The National Center for Missing & Exploited Children® was established in 1984 as a private, nonprofit 501(c)(3) organization. Today, NCMEC performs the following 15 specific programs of work, funded in part by federal grants (34 U.S.C. § 11293): Source: https://www.missingkids.org/footer/about
US DOJ OJJDP lists recent grants totaling $84,446,366 in FY19 and FY20. Source: https://ojjdp.ojp.gov/funding/awards/list?awardee=NATIONAL%2...
https://www.law.cornell.edu/uscode/text/18/2258A
You must report to them and only them.
For the GP to claim they’re not government “owned” is a rhetorical trick at best and outright ignorant absurdity at worst.
Imagine if Apple had done this on the client side without telling anyone, and later it was discovered. I think things would be a whole worse for Apple in that case.
In part because people didn't know.
And if you were one of innocent people caught by them, you wouldn't want people to know.
Conjecture, admittedly:
1. Apple cannot lose the Chinese market. Huge and more important fastest growing geo for the company.
2. China is self deprecating elements of its own tech sector (aggressive crackdowns on both established companies like tencent and Alibaba as well as individual web sites). They are clearly cleaning house from a surveillance and control perspective. Apple is not immune, but it’s an American behemoth, so open door crackdowns are impossible.
I don’t think Apple’s CSAM push and China’s crackdown are purely coincidental.
Who can argue with stemming child abuse? It’s the type of hot button issue that affords broad acceptance for intrusive tech.
The leap from scanning for abuse to scanning for anti regime content is more like a tiny step.
It seems obvious from afar that the company adamant about refusal to unlock a potential terrorist’s iPhone on privacy principles (with the attendant marketing benefits) would so suddenly force push (and therefore ensure collection massive training data with or without opt-in for Chinas v2.0) such a boldly invasive feature addition.
Turns out vertical integration is both gift and curse (dependent on the whims of the integrator) for on-device privacy and autonomy.
If Apple happens to use similar ToS in China (no idea if true), you bet CCP would be all over this clause.
A worst-case wild guess from the pessimist in me: they had to add this clause in 2019 to appease CCP, and then they got thinking if they could make use of it for the greater good (tm), too. Hopefully it’s very incorrect.
"Apple does not have my permission to use my device to scan my iCloud uploads for CSAM"
and
"This is a slippery slope that could result in Apple enforcing thoughtcrimes"
Neither of these viewpoints are particularly agreeable to the general public in the US, as far as I can determine from my non-tech farming city. Once the fuss in tech dies down, I expect Apple will see a net increase in iCloud adoption — all the fuss we're generating is free advertising for their efforts to stop child porn, and the objections raised are too domain-specific to matter.
It's impossible to say for certain which of your outcomes will occur, but there's definitely two missing from your list. Corrected, it reads:
"Either there will be heads rolling at management, or Apple takes a permanent hit to consumer trust, or Apple sees no effect whatsoever on consumer trust, or Apple sees a permanent boost in consumer trust."
I expect it'll be "no effect", but if I had to pick a second guess, it would be "permanent boost", well offsetting any losses among the tech/free/lib crowd.
"Apple has created a system for detecting CSAM on local devices which has already proven vulnerable to cheap perceptual hash collision attacks. It's now highly inconceivable Apple will be able to deploy this technology as-is without having their users exploited."
In other words it's not just about privacy or thoughtcrimes anymore but should be viewed as actually dangerous to use their devices. I feel a bit dramatic even typing that out but I.. think it's true?
How dramatic is too dramatic? When does something that hasn’t happened to you or anyone you know become a risk you’re willing to sacrifice personal convenience to mitigate? Will you be divesting yourself of all wireless radio hardware? If not, then why would you be worried about users being exploited through a more clumsy and less effective process such as CSAM signature hacking?
The piece of information you’re taking for granted, that few in free/tech/lib are confronting, is the assumption that this process can be exploited at scale to harm millions of people.
So far as I can tell, there will probably be zero or one false positive CSAM matches that pass the known algo, the unknown algo, the human blurred comparison, and the human unblurred comparison — all steps that must occur before law enforcement is invoked to collect digital evidence - in the first year.
How many false positives (to the nearest 10^X) do you think the system will generate in the first year that result in law enforcement actions? Your words suggest that everyone is vulnerable, and there are 10^9 users, so do you believe there will be 10^9 false positives in the first year? Do you think only a thousand people will be affected, so 10^3? How do you judge which is more likely correct?
It is unlikely that this system will generate 10^9 false positives, or else it never would have passed QA. I encourage you to consider how you would personally quantify this risk, and then also look up the quantified risks for killing someone while driving a car or getting struck by lightning while indoors. I don’t know what the actual reality will be, but I don’t think it's a very large X.
If I'm honest what makes me feel bad about this is probably just that- a feeling based on what I consider to be an algorithm designed around the presumption of guilt. It's much the same way I feel about taking my shoes off in line at airport security. It's an act which I've largely come to ignore but which still produces that vague feeling of discomfort that somehow feels like the opposite of security.
That this is occurring on my Apple devices - my favorite devices - is also just depressing.
I mentioned I was thinking of moving from an Android phone to Apple soon, somewhat privacy related.
My friends lectured me on "they're scanning your photos" ... meanwhile they share their google photos albums with me and marvel about how easy they are to search ...
Maybe we (humans) only get outraged based on more specific narratives and not so much the general topics / issues?
I don't know but they didn't seem to notice the conflict.
Psychologically, you'll feel a difference in what you accept between the two, I think
Don't want your photo's scanned, don't sync them to icloud. Seriously! Please include the actual system when discussing this system, not your bogeyman system.
"To help address this, new technology in iOS and iPadOS* will allow Apple to detect known CSAM images stored in iCloud Photos."
To increase privacy - they perform the scan on device prior to upload.
"for now"
Which is the part most people have a problem with -- they say that they are only scanning iCloud uploads now, but it's simple extension of the scanner to scan all files.
I don't care if Apple scans my iCloud uploads on iCloud servers, I don't want them scanning photos on my device.
I don't have a problem if it scans everything, but its not. Let's stick to what is doing. Android could do this as well, so talking about what companies like google could do is not so interesting - they could do almost anything.
I'm not sure there's a real difference unless you want to watch your settings all the time. In google land they tend to reset ... and really that happens a lot of places.
I think for most people if you use google, you're in their cloud.
No ... They scan everything that I have released to apple photos that exists on my device.
Same scan - different place.
I get to select the issue and it was in response to the previous claim that 'they' are scanning everything that exists on the device right now.
A year ago this was a possible 'future development'. 10 years from now I could be living on Mars. This is all hypothetical.
Exactly. However, people don't seem to have an issue when hypothesizing that CSAM detection is good actually, because Apple might implement E2EE, despite no evidence of such intentions.
For some reason, though, people take issue when others hypothesize that CSAM detection is bad actually, because Apple might expand it further and violate users' privacy even more. And there's actually precedent for this hypothesis, given Apple's actions here and their own words[1]:
> This program is ambitious, and protecting children is an important responsibility. These efforts will evolve and expand over time.
Must have been absloutely nuts round here back in 2011 when they announced the particular slippery slope that is iCloud backup.
Now we have clientside scanning, and actually being led by the [note the scare quotes] “pro-privacy” two ton gorilla. It’s a whole different ballgame.
But here appears to be beyond the pale such that people are saying "this far, no farther!"
How did you determine that their intentions contradict their words? Please share the framework for your belief, so that we're able to understand how you arrived at that belief and to evaluate your evidence with an open mind.
(Or, if your claim is unsupported conjecture, please don't misrepresent your opinions and beliefs as facts here at HN.)
This is just my biased view but there _could_ be countless ways to slip the slope without violating the current framing. For example:
Q: What happens when other governments ask Apple to use this for other purposes.
A: We will inform them that we did not build the thing they’re thinking of.[1]
Note that the question could be interpreted as "use this exact implementation" (rather than use the algorithm in general), and the answer does not rule out the possibility of "but we can build the thing for them then" (rather than "we don't have it and we will never have it"). The reader is free to interprete the conversation as they see it.
[1] https://daringfireball.net/2021/08/apple_child_safety_initia... citing [2]
[2] https://www.nytimes.com/2021/08/05/technology/apple-iphones-...
Definitely would appreciate a link to anything substantial indicating that this was a bunch of eng in over their heads.
Wow. That feels like an overreach. 'We don't just buy your labor, but also your private opinion and tell you how to talk to your family'.
I’m not buying the “engineers were left unbridled” argument, I think there just have been a level of obliviousness in much wider a part of the organization for something like this to happen.
Maybe because they underestimated people's ignorance.
I think they saw (and still do see) it as a better, more privacy preserving technique than what everyone else is doing.
Their hubris is in not seeing or thinking that they will be able to stand up to all kinds of abuses of this system that its mere existence will invite; their hubris is also in thinking that they will be able to perfectly and without mistakes manage and overview a system making accusations so heinous that even the mere act of accusing destroy people's lives and livelihoods.
Since the announcement, I can think of a dozen ways Apple could be easily forced into scanning all the contents of your device by assembling features they’ve already shipped. Yet they haven’t. At some point, people need to produce evidence that Apple cannot hold the line they’ve said they will.
What we have now is Apple, with its "strong privacy" record, normalizing this. If it succeeds, it would be that much easier for the governments to tackle other stuff onto it. Or, say, lower the threshold needed to submit images for review. I can easily picture some senator ranting about how unacceptable it is that somebody with only 20 CSAM photos won't be flagged, and won't somebody please think of the children?
And yes, if it comes to that, Apple definitely cannot hold the line. After all, they already didn't hold it on encrypted cloud storage - and that wasn't even legally forced on them, merely "not recommended".
Because privacy stance is mostly PR to differentiate from Google. And while there're invalid reasons to get users data, there're also valid ones (at least from legal requirement point of view - let's not get into weeds about personal freedom here and if the laws and its implementations need to be changed).
Their PR was just writing the checks they cannot cash without going on a war with governments.
I assumed they pivoted to focus on privacy, but clearly it was just a marketing department innovation rather than a core value (as their marketing department claimed).
If your threat model includes being the target of someone who will plant child pornography on your phone, you are already fucked. And no, Apple isn’t suddenly going to scan Chinese iPhones for Winnie the Pooh memes. They don’t have to. China already has the 50 cent party to do that for them, on WeChat.
Basically everything everyone seems to think is just around the corner has already been possible for years.
Most people won't have any idea about the meaning of "hashes" and "databases". Not everyone is trying to actively fight the system and shit on everything, most people just want to live happily with their friends and family, they won't care that Apple scans their devices.
> "Either there will be heads rolling at management, or Apple takes a permanent hit to consumer trust."
Oh god ! How wasn't all of this obvious to the top Apple management, but so obvious to epistasis! Damn, thanks man for correcting and leading Apple to the right track !
What I have seen is a selling point for apple products.
I'd encourage folks to get out of the HN bubble on this - talk to a wife, a family especially those with kids.
Why stop there? Get out of the HN bubble on the patriot act, instead ask your neighbor’s wife her thoughts on it. Get out of the HN bubble on immigration, go ask a stereotypical boomer conservative about it.
I think my sarcasm already made it overtly obvious but, this is horrible advice you are giving and the fact that you don’t seem to be aware that pedophilia and terrorism are the two most classic “this gives us an excuse to exert totalitarian control” topics betrays your own ignorance (or, worse, you are aware and just don’t care).
The only people who are bothered are people claiming this is going to be misused by authoritarian governments.
There are atleast 2-3 further checks to account for this.
Salvador Dali could do something similar by hand in 1973 in Gala Contemplating the Mediterranean Sea [1]
The mere accusal itself of possessing CSAM can be life ruining if it gets to that stage. More importantly, a collision will effectively allow warrantless searches, at least of the collided images.
And whoever going to check images for Apple will see that yeah, there is porn on picture. Flag it. Then you'll have unlimited amount of time to explain to FBI why some porn on your device match CSAM hash.
For a company that marketed itself as one of the few digital service providers that consumers could trust, I just don't understand how they acted this way at all.
Either there will be heads rolling at management, or Apple takes a permanent hit to consumer trust.