They just gave a talk at the USENIX Security Symposium on how this works and the safeguards put in place. The scanning functionality is part of the icloud photo upload module. Security researchers can determine if they change the scanning algorithm in future updates. I don't know what else you can hope for.
Going back to the slipper slope argument, you could say nothing stops them (or Google on Android) from uploading your passcode and share it with X. It's all just software in the end that they write that powers the lock screen and security checks.
If they start claiming they do that then I'll get angry about that too. You'll notice that only the fringe is accusing Apple of being liars or acting in bad faith here. Apple are pretty up front about what they do.
The problem with this plan is they're claiming they are only going to do selective law enforcement (only 1 US law, only things that are strongly supported by consensus and even then only sometimes). That isn't a position with a reasonable foundation, they are going to change their mind. I want them to change it in the "we don't snoop on people's phones" direction rather than open season.
Again, I don't expect them to do this. But they could add a useful level of trust if they were really motivated to do so.
As for no company giving up rights, I did say I didn't expect Apple to actually do this, only that they could. Though, companies do actually make binding contractual promises all the time. It's just that they don't tend to do it with consumers or small businesses who have no negotiating leverage and/or no desire to insist on those promises.
Regarding your hypothetical, anything that's actually legally mandated by the government generally overrides contract law since the courts won't enforce illegal contracts and illegality is a defense to breach of contract. They can always do what the law actually requires.
But we're not talking about the government mandating things, since both the iOS 14 behavior and the iOS 15 behavior comply with the law.
Which means this action is just a charade, as I pointed out. Because fundamentally it doesn't really matter. The gov is going to do what they do, and Apple is not going to create a situation in which an external group can determine how or if they can be sued in court based on how overreaching they feel that day.
Also go check how many arbitration clauses exist in your ToS, just as an exercise. I'm all for banning arbitration clauses and allowing consumers recourse in actual courts. I don't believe we should have companies sign their death sentence prior to doing business, this raises the barrier to entry and results in even less competitors who have to do the same.
I could hope for being able to trust that an intensely personal device such as a smartphone isn't something that I have to be constantly suspicious of.
Admittedly, that ship sailed a long time ago, but I could still hope.
You being able to trust them is a personal choice. It seems people are happy to trust companies that don’t talk about what they do with their data rather than the ones that do. Which, while understandable seems counter intuitive.
Could they write and deploy something overnight that hoovered up everyone's data? Maybe on iOS, less so on OS X, but now they're going to ship with that capability. I don't understand how some people can't see the difference between "they could always push some nasty update," versus literally shipping hardware and software with a backdoor.
But it would be very hard to spot new weights of the neural network and a new list of target hashes. These are pretty much guaranteed to change regularly as they retrain the embedding network and/or change the list of known target images. So it will be very hard if not impossible to see what they're searching for. That latest update could just add the ability to recognize CSAM pictures that had meme texts added to them. Or it could change the embeddings and target list to spot unlicensed posting of copyrighted still frames from movies. Or it could now retrieve any picture of people in police uniform. No way to know if you don't have a hunch and a targeted picture you want to test with.
Do you know this for certain about any vendor? If a company the size of Apple were pushing the same update to everybody, then it would likely be known about by the world pretty quickly. But... if a targeted signed update is sent to a handful of selected devices, that's harder for the world to find out about. It's risky, but it's definitely technically possible.
You cannot be serious.
Let’s compare:
— Apple’s market cap is more than twice of Facebook’s.
— Apple’s user base is less than half of Facebook’s. Even fewer use iCloud Photos (and much fewer upload more than the free 5 GB, shared across photos and other content).
— Facebook is entirely made of UGC with no storage limit for a given user, and their moderators have to review every bit of content to ensure the gore is away from advertising targets’ eyeballs (and, indeed, report CSAM). Apple only needs to review images that trigger multiple hash matches occurring within the same account.
Considering the above, my intuition is that Apple’s NeuralHash would have to be completely broken for a company this size to not be able to afford enough moderators to manage the false positive rate. I sincerely doubt they are so inept, and I’d be willing to live with the potentiality of an Apple employee peeking at a photo of mine once a year (me using Apple tech already implies I trust them enough, as I’d never be able to personally verify every privacy claim they make).
What is worth screaming about, and what is eroding my trust, is the fact that since 2019 Apple’s ToS (quietly changed, presumably, to accommodate this feature currently in the news) give them carte blanche for pre-screening any content that they deem potentially illegal. Unless they tighten up that phrasing to limit it to CSAM only, they’re allowing it to be used as a political prosecution tool.
Edit: Factual error, iCloud Photos does have a free tier.
Anyone trusting Facebook with a level of access to their life comparable to a mobile phone is foolish. There is no way I would pay money for Facebook to control my phone.
If it costs them too much money to have too many people personally looking over every single positive hit to make sure it’s not false, then they’re not going to do it even if they technically could.
I live in a place where the government arrests you for having a VPN installed on the phone and labels you a terrorist outright. My phone is checking with physical frisking on the roadside and content critical of state gets automatic manhandling and trip to the police station where I am treated as a criminal.
How I see this as a problem not because I am not going to buy an iPhone in future, but because such ideology would be made normal. That is what is scaring me
The technical implementation is trying to hide the fact that the design mojo of this system is an actual Backdoor for governments of the world to oppress, censure and do whatever they like.
The technical implementation is optimized towards minimizing the cost for Apple. That's why they scan on the device. And this decision is made with knowledge that in the near future "scanning" will be not limited only to hashes. Scanning and processing will be required for $Some_Cool_Functionality to work.
The same slippery slope applies there.
It's Sir <firstname> or Sir <firstname> <lastname>.
https://www.geni.com/projects/Naming-Conventions-for-Knights...
1. I don't want to financially support anyone going through my private things in conjunction with what is basically the police, looking for reasons to imprison me. Reasonable suspicion first, thankyou very much.
2. I don't trust them to use this in a politically neutral way. This is too much power.
3. I don't trust them to manage the false positive rate. Everyone knows how reliable software engineers are when they claim a system does something. At heart they aren't 1 in a trillion people.
This is a great time to be outraged. I can't really do much about what they do on their servers, but I can certainly get antsy about what happens on my phone.