It is super well documented that Apple ALWAYS follows the laws of the land, so the scenario is that a government adds some new hashes that might not be CSAM as you and I define it but might be political stuff(but the citizens can't check). Also now that the code is there Apple can't refuse to add this hashes or enable the feature forcefully when a legal warrant for that territory is presented.
some more config changes and it can start scanning for text in documents too.
Here's what would need to change about the system to fulfil your requirements.
1. The local government would need to pass a law allowing such a change 2. The hashes would then need to come from the government. Instead of the two intersection of two independent CSAM databases. 3. The review process would need to redesigned and deployed to provide a lower threshold and permit the reviewers to see high resolution images. Neither of these changes are trivial. 4. The reports would then need to go to the government. 5. What's stopping these same governments from requesting such things from Google and Meta - both of which have comparable systems with lower oversight?
Apple don't "ALWAYS follows the laws of the land", one can read into this with the recent "secret" agreement between Apple and China which detail the various ways that Apple hasn't complied with Chinese requests (e.g. the requests for source code.)
Sure, it is not hard, but already laws can apply, did China had to pass a law like "any company has to update their maps software to conform with our local maps?" or China just showed Timmy a graph with profits from China dropping and then Apple said "yes sir!!!" .
> 2. The hashes would then need to come from the government. Instead of the two intersection of two independent CSAM databases
Yeap, the government will demand for a local database to be used and since the crimes are local you need to send the reports o the local police and a local Apple employee will have to check, not some cheap random contractor.
Apple's response to the "what if governments force you to abandon the hidden set intersection stuff you wrote" thing is, "well we'll just say no". Which, unless Apple plans to pull out of hostile jurisdictions entirely or levy their own private military[1], holds no weight. Countries own their internal markets in the same way Apple owns the App Store market.
[0] https://en.wikipedia.org/wiki/Restrictions_on_geographic_dat...
[1] which should scare the shit out of you even more than the CSAM scanner
The CSAM scanner does not scarry me as it is today, but the fact that is a step, next year will scan more files, then Google will copy Apple (willing or not) and then most PCs will do the same thing.
Apple just normalized that is fine you have an proprietary file scanner/"antiviruys" installed on yoru device with the properties:
1 it can't be stopped or removed
2 it get's hashes from some government agencies over online updates
3 matches will be reviewed by someone that it said are trustworthy and might get sent to your local police
4 the stupid scanner has stupid excuses to exist, where would have been extremely simple to implement and non controversial to scan the fucking images after the upload and not before.
If it's triggered, send the details to a particular group of individuals with certain clearance to view and verify.
If they have the ability to do this, they will definitely do this!
Of course they could decide later to implement it! Just like any company could choose to do that! It’s the whole point, your phone is closed source. But clearly apple didn’t pick that route, they announced it BEFORE shipping and asked for feedback! And you’re blaming them for that?!
Isn't obvious ? Some guy using an exploit would install a debugger on his iPhone and find this scanning process, Apple PR would be hit. But Apple could do this and gain PR by pretending is a safety feature, the guy with the debugger will see that files are scanned and hashes are sent to the server but he can't prove that the config file won't be updated in the future for everyone or some and the hashes will be pulled from a different database, the threashdold will change and the is iCloud enabled flag will be ignored.
What makes no sense and Apple failed to explain and fanboys also failled to convince me with any evidence, why the fuck not scan the images on iCloud only since we are pretending that only if iCloud is enabled the images are scanned. (there is no credible evidence that this is part of any plan to offer some kind of super encryption that Apple has no keys for)
"See that hashing thing you have, this secret gov. order that you can't tell anyone about says you need to look for this list of hashes too. If you find them, you have to let these 5 people know and if you tell anyone you'll accidentally shoot yourself three times in the back of the head to commit suicide!"
Didn't that only happen after a leak?
The vague "this is a serious concern because my government is shit" requires a person to ignore a lot in favour of a little.
I think you missed OP point, the grandparent found a bad excuse for Apple, that Google and Facebook do the same so nothing is wrong, but the OP responded correctly that this is false, those companies don't install file scanners on your device so the excuse is bullshit.
They don’t install file scanners on iOS because they cannot. They are happy with scanning my contacts and doing who knows what with my photos if I let them, though.
Nobody says that Google or Facebook are good/saints, but there is a difference, my phone did not had Facebook or WhatApp installed, so journalists or activists can just don't install this crap.
> Then you should already be gravely concerned
Reverse this for a moment: would it be acceptable that a Chinese tech behemoth decides US law does not apply to their American operation, because they are convinced that western democratic values are dangerous?
This is also a real concern. If you think that rule of law is a good thing, then companies must comply with local laws. The alternative is not a humanist utopia; it is the rule of unaccountable corporations.
In this case there is no law to force Apple to scan on device, they can scan iCloud only if the law forces them to do that, or scan only iCloud images that are public or shared if the law forces to do only that.
Before 1989 listen to a specific radio station was illegal , but the good part was that the radio device could not snitch on you , if someone would have invented snitching radio then I am 100% only those radio devices would be legal to be sold in my country at that time. So if you create a snitching device then you invite the bad governments to add the laws after to profit from that and make their lives easier.
I don't think that offers meaningful protection. A malicious government could acquire or produce real CSAM, or at least a convincing fake of it, then modify that image subtly to match the hash of the political content they were targeting (I believe such engineered collisions have already been demonstrated.) They would then be able to get that manipulated CSAM into as many independent CSAM databases as they wanted, simply by uploading it to the web then reporting it to all those independent databases.
1. Forced hash collisions are fragile in comparison to the genuine material. CSAM hashing is devised in a way to allow change in the image without significant variation to the hash. Forced collisions however don't possess that property as their true hash is different.
2. It's trivial to detect images with false hashes, and different hashing methods will not produce collisions.
3. The task you're proposing, in itself is not pragmatic nor effective: dissenting political materials are easily created fresh (i.e. no colliding hash) and exists in many forms that go well beyond an image.
4. Human reviewers again would see the pattern and adapt the system accordingly
5. CSAM database holders aren't idiots. They understand the significance of their hash libraries and work against manipulation from CSAM peddlers.
Point 2 and 5 I don't understand, could you expand on it? If the government produced manipulated CSAM that didn't exist in any other pre-manipulation form, how would the manipulation be detected? And supposing it were detected, would CSAM database holders choose not to add that image to their databases just because it was manipulated? If that were the case, it would give pedophiles a way to exempt their images from the database too.
Point 4 is neutralized by the government demanding local human reviewers for phones located in their country. China already does this with iCloud, they could demand the same for the CSAM reviewers, then coerce those reviewers or simply select them for party loyalty in the first place.
And it hasn't eventuated. But yet now it is something to panic over ?
OSX is getting more and more locked, iOS is locked so why shouldn't people complain when things get worse.
I was just explaining that the excuse "OSX is 20 years old and it was not locked yet so it will never be locked q.e.d " , this is wrong, you can see clearly how things get more locked in steps and you don't have to wait untill last step to complain.
Keep in mind that you have better keyboards on Apple laptops today because people complained, the CSAM shit was delayed because people complained, the Apple tax was lowered in some cases because developers and Epic complained, so complaining sometimes works.
Yes, I agree that your phone has the ability to find stuff like this based on ML etc. However, if their algorithm finds something that matches a hash of child porn, you'll be reported (after notification or something, can't remember).
Can you imagine what that would do to people? The risk of fucking this up and destroying someone's life is far too high. It's a stigma that will never go away even if it's proven false.
And what if the government decides that they want to ruin a political adversary? The technology is there to report them, rightly or wrongly, for child porn on their phone and ruin them. Again, proving that it was false will not be enough if it leaked out in the first place.
In my opinion, it's my phone (yes, yes, I know they all dial home to the mothership, I only lease it etc.) and I should have complete say over it and this is one more step away from that goal.
It's on the dystopian highway imo.
After 30 hits (so 30 false positives), and then the photos are manually reviewed by a human being. How did people never understand this? How on _Earth_ would people's live be randomly destroyed by this? The reviewer would take one look (or 30) and immediately realize it's not illicit. Crisis averted. But the chances of those 30 false positives actually all being in someone's iCloud was like astronomically unlikely.
> And what if the government decides that they want to ruin a political adversary?
Then you're already fucked, and the child porn scanning software won't be what makes it possible to ruin your life.
Don't be naive, the governments will demand that matches on the "special" hashes will be reviewed by local government people.
There is a similar feature in the "child safety" package that scans for stuff in chat messages and probably (maybe soon) in documents that are received ), so the code is in place to have an uninstallable "antivirus/antiCSAM/antipiracy" on your device that you must trust both Apple and your goverment that does what is promised but 100% you can't just delete the executable to have piece of mind.
Seriously, if a government is after you, they will get you regardless of the content of your phone. In a country with decent laws, just a match is woefully insufficient to demonstrate someone’s guilt, and they will need some proof. In other countries, laws do not matter so they can just get you without issue. They have done it for quite a long time and are doing it today.
All of this also points to the complete absurdity of this mass hysteria: it is insane to ruin someone’s life just because they have a picture on their device. It is just thoughtcrime, at this point.
But it happened, some guy was falsely accursed because someone made a typo in an Ip address, he lost his job and he was forcefully kept away from his wife and children.
>Seriously, if a government is after you, they will get you regardless of the content of your phone
That is false, for example before 1989 listening to a certain radio station was illegal here in Romania but the government could not ask the guys the made the radio to give them a list with all people that listened to this radio station. Your claim is that the govberment knows my name and what I did they can fuck me up, that is true but this imnplies the gov already knows and has the proof, in this case without the file scanner they won't know I own political/religions materials that is illegal but after the file scanner runs and reports me , only then I am on the gov list.
I hope is clear, tech makes the job of the bad gov easy, so in this case you as a random user you gai nothing from your device scanning for CSAM but many lose from this, the question is why if you gain nothing and this can be done with a simple iCloud scan? (because for sure it will be extended to private messages and documents)
This makes it seem like there is no upside at all to the CSAM scanning. There is: we stop more folks peddling child porn, hopefully making a dent in the problem.
Apple could have scanned iCloud from the start and prevent this problem if it is so big. If they are already scanning then there is no use for scanning on device only if iCloud is on and if they are not yet scanning iCloud then you should ask Tim why is he ignoring the problem, Google and Facebook already reported a lot of abuses, does Tim love CP?
You can guarantee Apple will report you. They have to. It will be the law.
It will be used for human rights abuses. Guaranteed.
Is that an issue? Sure! Is it a new issue, or specific to Apple's implementation? Nope.
This is already true in certain circumstances.
> then the entire tech ecosystem we all use everyday is completely untrustworthy.
This depends on what tech you use. The degree to which you cannot trust it, or need to understand that it is actively working against you, varies by platform. Nearly all platforms undermine you in different ways. Some of those platforms do so in a way in which you're ok with.
> Is that an issue? Sure! Is it a new issue, or specific to Apple's implementation? Nope.
Agree to disagree. I don't mind cloud services (someone else's computer) scanning my data and working against me. I do mind my computer scanning my data and working against me.
Not too many people get caught, review team gets downsized since it is not a revenue generator. Maybe we can replace it with ML!
How many stories do we have on HN about overworked employees reviewing horrible material? Or about how impossible it is to contact support?
Just because there is a good system in place now doesn’t mean it will stay there.
As much as I think the panic is way overblown, this bit is simply unacceptable, and they need to get nailed just for having thought this was a good idea. Nobody is to review personal photos; this is an insane breach of privacy.
Many people here simply assume all governments and corporations are simultaneously that evil and that incompetent. The aggressive cynicism of this forum and its reticence to engage with content before commenting undermines its pretense at informed technical acumen and civil discussion in many ways.
Sounds like a great gig for a pedophile.
You do know that Apple is scanning your photos server-side, right ?
Just like Google, AWS, Yahoo etc anyone that stores your photos in the cloud.
Yes.
They have every right to. It's their servers.
However, this is MY phone (ok, not mine, I don't use Apple devices but you get my point)
I fail to see how this system (which I'm not a big fan of, to be clear) makes it easier if you assume governments can and will fabricate evidence. Doesn't seem you need an iPhone, smartphone, or to have owned or done anything at all in that case.
The objections come when such systems are to be run not on corporate property, but on end user property. I don't see anybody objecting to corporations scanning content uploaded to the corporation's own computers. Scanning content on the user's computer, which is the user's property, is what unnerves people.
Furthermore, while Apple may have implemented this feature in a relatively safe way (I don't fully understand it so I'm not confident of that, but smart people here think it so I'll assume it for the sake of argument), I believe that the general public does not really understand this nuance and Apple normalizing the practice of scanning personal computers will clear the way for other corporations to implement scanning in shittier way. If Microsoft were to start scanning all documents on a Windows computer by uploading those documents to their own servers, is that a distinction that would necessarily be made clear to the general public? I don't think so. The rhetoric will be "Apple does it and that's fine, so Microsoft can do it too". The technical differences in the two implementations will be understood in forums like this, but not by the general public at large.
For example Google and Meta both scan all material that is on their services, both of these providers actually go further by utilising AI to hunt imagery which may not be identified yet. So the idea of this compelling others to scan is false: Apple is last here.
Similar to Google and Meta, Apple's CSAM processes only come into play when the material is uploaded to Apple's servers. The minutiae of where the scanning occurs is irrelevant because Apple's implementation is merely a way of performing the function without giving up the user's privacy (by scanning locally rather than sending all of your photos to Apple's servers for review.)
The scanning process is merely the addition of verified hashes to a system which already uses this same technique to categorise ones photos into searchable images ("waterfalls" "dogs" "bicycles" etc.), because this is an existing system there is no impact to performance/device usage.
This is where you and I part ways. I don't think that's an irrelevant minutiae, because it normalizes on-device scanning. This implementation or others may change in the future and opposition to those future implementations will be hampered by "Apple was already doing it" arguments, which will sound fairly plausible to the non-technical public.
Maybe I'm making a 'slippery slope' argument here, but I've become comfortable with such lines of thinking. My cynicism has been rewarded too often before.
That depends on how "BYOD" the company is. There's an awful lot of companies that provide work machines that most of their employees reasonably use for some sort of personal work, be it checking their bank statements to see if they got a deposit or logging into a handful of services so their settings sync. In my opinion it is unreasonable to expect people to not do this, especially given that the alternative is quite painful with modern software. Corporate searches have a problem of not really having clear limits like a legal search might, so going through e.g. your synced web history when trying to find evidence of credential abuse (check your own audit logs!) is common.
I feel the fears about this implementation are largely overblown - equivalent features are not being misused on Google and Meta/Facebook's properties, despite having a much larger audience and less procedural oversight. Rather Apple is the outlier here for being not just late, but also by having minimum thresholds that must be passed and an unskippable step of human review.
I'm yet to hear a coherent argument about the potential dangers of the system that understood the implementation, nor the state of image scanning amongst cloud providers - even this thread seems to believe that Apple is the first here. Apple's approach was the most conservative, Google's goes much farther: https://support.google.com/transparencyreport/answer/1033093...