BTW: I've been an Apple fanboy since the mid-80s, but I find the whole concept of my phone suspecting me of child pornography appalling. It's the thin end of the wedge, and IMHO Apple have made a huge mistake.
This is the biggest worry. Apple essentially waved their hand in their FAQ and said something to the effect of they’d never do something like that.
Yes you would. You’ll get called to congress and told in no uncertain terms what you’re expected to do, you’ll say no, the law will be changed, then the gentleman in a suit that people don’t say no to will call to your nice headquarters for you to walk him through his shiny new surveillance tools.
Once the tool exists, they’ll add whatever features governments in major economies tell them to or they’ll be forced into adding them.
All this said, I don’t blame Apple alone here and I have mixed feelings about it still - as I really do believe in their fight against CSAM. I just don’t think this is the magic solution to solving CSAM while preserving an innocent user’s right to privacy that they’re selling it as.
I’m also certain there are engineers who worked on this system who also share these concerns.
I also feel it's important to note that every major image host is doing the same thing Apple is doing, but server-side on unencrypted images. What Apple is planning to implement is, out of context, an improvement; in the same way that Google's FLoC is an improvement over ad tracking based on third-party cookies. Get a trusted client to cooperate in some smaller act of self-surveillance and we don't need the server to do full take.
That being said, I don't think we even need to rely on governments threatening Apple into doing surveillance. The scanning scheme relies on paying human moderators to basically look at CSAM all day and check if it's a valid match. Big tech moderation is already understaffed and overworked as is - so it's almost certain that the people involved will just forward everything as their sanity degrades. Apple will almost certainly wind up accidentally or unknowingly forwarding along bogus CSAM reports, no coercion required.
In The Cuckoo's Egg, Cliff Stoll tells about having his computer page him when it detected a hacking attempt. Telling about one time when his pager went off, he said, "My computer wanted me."
Now we have so many notifications and suggested actions from our phones and computers, it feels like we have become computer peripherals. Our computers want us to do things. They have agency, and we're responding to it.
But at least we still had agency, too. The computer had the initiative, and we could choose to respond, or not. Now, though, the computers are limiting our agency. They are becoming the controllers, and we the controlled.
I don't think it's just that Apple has made a huge mistake. I think humanity is making a huge mistake. We are allowing tech to turn us into children.
It's been a decade and a half of software not being the user's agent and instead some company's agent that in some cases you buy the hardware for.
Unless how we talk about software changes this is just the world we live in now. How many apps do you have that are genuinely extensions of your intent as a user and not company kiosks? Of probably about 100 apps on my phone I think I have maybe 5.
My biggest fear is the only reason I've seen this come up this week is the Apple community, that built me into a programmer from a waiter, has become susceptible to the same kind of irrational tribalism we see everywhere else - "surely this bad thing is happening because it's part of a secret plan to fix the other bad thing!"
Another article recently points out that even the md5sum databases include bad matches (a rhesus monkey was mentioned).
Not everyone is able to do this, but you're technical enough to be spending time on Hacker News.
(Also, this doesn't preclude lobbying for restrictions on government and corporate surveillance. It's not a dichotomy)
On my desk right now there are five Android phones (two of them don't even have SIM cards). I use them for different things. I wouldn't dream of signing in to all of them with the same Google account.
If you want to retain some degree of privacy and use Instagram on your phone, you need to get another phone for that.
They're cheap.
Second, phones are a terrible platform for privacy preserving technologies, both currently (mobile OS dumpster fire) and intrinsically (lower resources, poor input, well-known network identity). Anybody who cares about digital privacy should have a real computer laptop/desktop that they use for the bulk of their activities.
Not being locked-in into apps is critical nowadays.
Someone earlier astutely stated the metaphor “the thin edge of the wedge”; the movement to the “cloud” via the open web over the last decade was the thinnest part of the wedge.
Even to the extent web is open today, I’m able to run Linux on all my PCs/laptops and not to have an extra one to run proprietary OS in order to participate in modern life.
If we don't want to live in a surveillance-based dystopia, we need to push back against it politically as a society, not play cat-and-mouse as individuals.
And for that to work, we have to counter the usual "but think of the children!!!" narrative with one of our own - prevent CSAM creation at the source. I have written about how to get started previously: https://news.ycombinator.com/item?id=28114076
I think one can arrive at the opinion they would prefer not to have their recommendations flooded with stuff from from alt-right or conspiracy Youtube/Twitter/whatever without buying into propaganda.
Meanwhile, groups/people/movements that completely match that description are real and it's handy to have a term for that behavior, even if you think it's applied too broadly (and maybe it is). We've got the similarly-abused word "cult", but abuse aside it's still handy to have a way to refer to groups that actually do closely fit the colloquial, pejorative meaning of the term, and there's no way you're going to get people to stop using the word that way.
It should disappear due to active push back because freedom, democracy and human rights cannot coexist with such surveillance.
Privacy is where independent thinking is happening and generated and if it's not guarded properly the whole idea of democracy is not possible. More on that here : https://news.ycombinator.com/item?id=28084578
But to convince Apple there needs to be plenty more articles about CSAM and where it leads.
Everyone here is hating on Apple, but looking at what Peter Thiel is involved in - oh, that's FINE. He is rich, he is better than you, obviously. Don't believe me? Watch the Peter Thiel shrine that is HN downvote this.
https://www.vice.com/en/article/5db4ad/google-bans-safegraph...
Apple has just created a product that slowed the government down and they wouldn't stand for it and so they compromised.
But America talks a good talk about pushing back against surveillance but if it doesn't affect people day to day, no one will care.
Hell, almost a million Americans are dead and about half care more about being able to go to a bar than the easily preventable deaths.
Corporations have done nothing but repeatedly invade my privacy, exploit my data and attempt to squeeze money out of me at every turn. The relationship has seemed adversarial for quite a few years. So what should i expect from Apple here?
It sort of feels like the alternative is for Apple to stand up to governments. To fight for the people.. But this seems super, super hopeful. A desperate plea founded in wishful thinking that corporations have any incentive aligned with ours.
So what in your eyes is the right solution here? Apple letting us down here is par for the course here to me. But i also don't see an alternative where i'm not just blindly hoping some corporation will defend my freedoms for me. I've lost all trust in literally every other corporation.. honestly, should i have trusted Apple?
Today it's child pornography. What about tomorrow? Let me make a case. First, let me make something clear at the outset. I am 100 percent against child pornography. For all I care, castrate all the adults involved, when you catch them.
Now, with that made clear, let's take a look at how bad, comparatively, pornographic photos of underage individuals on a person's phone is—and my apologies to those who cannot stomach the following discussion.
The photos have already been taken. The harm here is indirect. The harm is that possession of photos means that you have become part of the market for child pornography, and provide incentive for the production of future materials. The direct harm occurs when the materials are being produced, and the direct harm is categorically worse than dissemination and possession of the photos.
Compare that to something like terrorism. Recruiting terrorists and planning terrorist activities is, I would argue, a greater and more direct harm than mere possession of child pornography. (Leave aside the production.) So, how can someone accept that this is okay for child pornography and not terrorism? And once we've accepted it's okay for terrorism, who is the "potential" terrorist? What is terrorist contraband?
And now that we've accepted sniffing for terrorists, what's next? Election "manipulation" that "threatens" our democracy? Public health "misinformation"?
This is the Panopticon.
Our latest Antipope.org post (discussed here yesterday), suggested we extend the child-pornography treatment to possession of unauthorized cryptocurrency.
https://www2.palomar.edu/anthro/animal/animal_3.htm
How could anybody be against criminalizing it!
Interesting. I’m not familiar with a criminal justice system with a false positive rate lower than NCMEC’s perceptual hashes.
I doubt it would suffice. Chemical castration, perhaps.
Is this a viable option when governments are pushing them to do something? To give governments access, etc?
Ie why should Apple fight governments here. They seem to have next to no incentive to do that, in the capitalist sense. Our society doesn't typically financially support moral stances, so is fighting the government, which "doing nothing" surely is, a realistic stance? It seems insanely hopeful to me.
Implement encryption and say the same to the FBI the last time they wanted apple to unlock something: Sorry, we can't.
It's not like encrypted cloud storage without holding the keys or client-side scanning would be new ground. Providers for that already exist and have existed for years.
I don't really agree with this system either but I, at least, appreciate the sentiment that they're trying to solve this problem while also providing E2E encryption (assuming that the internal info is correct and the end goal is full E2E encryption).
Or activists, journalists, doctors, lawyers, victims of police overreach, etc. If this is a question of framing then shouldn't apple be more than qualified to handle it?
Honest question, were are you going that has less privacy issues than this?
I _want_ to support some Linux-only phone, completely off of Google/etc. However, i suspect that will be too poor of a UX, lacking basic functionality that i deem essential. I may still buy them to support them, but i think i'm going to have to buy a Android-based phone and then heavily de-google it. I don't like this option because it still supports the Google ecosystem, but it may be a necessary evil while the Linux-phone market hopefully grows, and while i continue to buy the Linux-phone market to "do my part".
I've flashed mine 2 weeks ago and I am very pleased. Without the Google Services Framework you are restricted to the F-Droid store though (not a problem for me personally). And there are several Apps that won't work even if you install the GSF and the Aurora-Store: Banking-Apps, Online-Check-In for Flights, etc. -- again not a problem for me personally, I fly maybe once every 5 years and use Online-Banking from my browser...
HN is a weird crowd, so this obviously isn't a solution for +99% of people. But I'm choosing to opt out entirely. Living in the 90s is annoying when coordinating with unreliable people, but this isn't some kind of impossible feat. I don't even think about it anymore, it's a non-issue. Just get on with it.
They are sitting on ~$200B in cash. If they want to continue to be viewed as protecting users data and privacy, that is exactly what they should do. Spend some of that money in court, defending the privacy of their users. But they won't, and they will lose the thing that differentiates them for some subset of their users.
Yes, it's hard, cumbersome, and not "there yet" for a long time to come on a lot of fronts, but it's the only way out of the corporate multiverses.
Years ago it was something that only nerds and techy people would use and now one can easily install it for any family member and never look back.
How about not moving forward with this system?
Corporations have to obey the law, but they don't need to be zealous and build spying infrastructure when they aren't required to.
(and all of that would be before I even consider false positive rates)
But even if that was the case, the tool will now exists for any dictatorial government to spy on "troublemakers". The Chinese government, which has already gotten full compliance from Apple, would be foolish not to use it.
On one hand it's encouraging to read your take, but I have no faith that in another 10 years, the government scanning all our devices for "misinformation" that the then regime deems as such, will be anything but hand waved away by the majority of SV employees as "it's the law".
> Apple just side-stepped the question on how to reconcile trust in propietary software and surveillance requests from the government.
They actually inserted themselves into the process, not side-stepped it. They could’ve kept status quo (no scanning) and kept supplying iCloud Backups to governments at large rates. As part of this, they’re also forced to add a greatly increased human review function.
> We don't even know what big brother wants to know about you.
They’ve explicitly stated what they’re scanning for and went further to say they will not scan for other types of content. You may not believe them, but this is not a content-neutral or blind justification.
Given pressure from governments for this type of surveillance is, i think, well agreed upon - Apple giving in while avoiding direct responsibility seems quite the incentive in my view.
For sure there had to be a lot of pressure. I don't believe Apple did just because it was a beautiful morning once.
I'd guess they tried it once or twice, and it was too slow so they went for more forceful means.
This just reasoning from what they've already done in the past.
That’s a very bold claim to make on hearsay. Does your friend work at Apple? Are they just speculating, or do they have actual evidence? Why do you believe them?
C’mon, people, I know this is a knee-jerk issue, but exercise some basic critical thinking skills.
As far as the attack goes, I have a few more details, but I'd rather no reveal them since I could be violating confidentiality somehow. I understand why you'd be skeptical.
Winning!
To follow the law? You can already read a bagful of stories where it caved in on law enforcement pressure. It already follows much invasive legal systems in China or India like any other major tech company. The measures there are being presented as "anti-terrorism", maybe a less marketable term in the US currently.
Wait a second, but only if you're uploading photos to iCloud? They were presumably already doing this once you upload them, so what is the big difference? I have totally missed this detail in a lot of the coverage, and it changes my view of it.
I'm aware the general sentiment here is "don't assume governments are putting pressure on Apple to step up their reporting, this is all on Apple bad bad bad bad bad," but sorry -- those numbers do indeed lead me to strongly suspect governments are putting pressure on Apple to step up their reporting. And I don't think Jason Snell (the author of the linked article, who probably knows more about Apple's world view and internal politics than almost anyone outside the executive team does) is wrong to theorize that Apple sees this CSAM-hash-detection system as a way to meet that pressure while protecting user privacy as much as possible, because they've convinced themselves that "protecting user privacy" means "do as much on device and as little in the cloud as possible".
To be clear, I don't think this is a good idea -- letting the camel's nose in the tent through a different opening is still letting the camel's nose in the tent. The irony, though, is that I think Apple's inventively bad solution comes from the most canonically Silicon Valley mindset ever: all problems are engineering problems.
I think we have to look at this through the lens of governments around the world attempting to ban E2E encryption. E2E encryption has both security and privacy benefits, and IMO most people are far more affected by the security benefits, given the frequency of data leaks/hacks. So maybe Apple is saying "let's preserve some of the E2E encryption benefits", but show it can be compatible with LEO demands.
I think the unfortunate reality may be that that is the least bad option.
But I think your arguments would be improved by actually reading and understanding the current system Apple is proposing. Which, for example, does not include automatic reporting to authorities based only on a hash detection.
At the end of the day this is mostly an issue of political and legal requirements. Apple is doing what they think they need to do to cover their bases as they wrestle with governments over things like E2E encryption policies.
A utopia of GPL-only mobile devices with no proprietary cloud service connections doesn't seem like a valid option. At least not for a large enough share of users to make a societal difference.
Nah. Everybody else will live in that world because they are addicted to convenience. Personally, I'll be scrapping my iPhone and buying something with free software.
Apple's the first to do this on your own device. Sure, only content that would have been uploaded to iCloud. For now. But this is the start of that particular slippery slope.
Any form of automation and convenient goes against the main point that is privacy.
If you share cryptomator and the vault to your family/friends they can access it but it's not possible to look at the pics on the web that way.
The app is also paid but it's a one-time fee and not expensive.
I don't know what Apple is thinking, but I doubt it's that, because people who care about being surveilled are very unlikely to be persuaded by that argument. (And obviously people who don't care much about being surveilled are unlikely to care very much whether such surveillance happens on device or in the cloud.)
Edit: I guess at worst Apple's just sending a cryptographic hash of unencrypted data so that the known childporn can be flagged. It's basically like sending a hashdeep record of the data. It's an interesting concept with at lot of implications for tracing the spread of files within networks without knowing what they contain. Presumably if it was expanded to all files in cloud storage then LE could say something like "Give be a list of everyone that has object XDFAECXDZE" when LE knows that XDFAECXDZE is Catcher_in_the_Rye.pdf and Apple does not.
My understanding is that this only works for known abuse images and even then only if the files are identical. So apple has no way to know if high schoolers are sending nudes to each other.
No, we don't. Your refusal to understand how any of this actually works, your desperation to conflate this process with the worst hypotheticals, does more damage than Apple ever could.
I have an old razor flip phone I might start using again. That and an iPod touch with no cellular.
This is here to stay. Time to adapt to it.
This is straight up a lie. Comparing hashes is not anything crazy, nor is it an API for big brother
As implemented, I agree with Apple
That's the point. It's not an open to the public list, it's secret and controlled by few. It can contain whatever they want it to contain.
> Not how this system works.
That's absurdly naive.
Also, I'm willing to sacrifice a tiny bit of privacy to stop child porn collections.
The hashes are then sent out to for comparison to the hash table.
Right now they are only comparing to the CSAM table but they can add other later.
Also, how are the hashes sent? Can they be intercepted?
If the Apple reviewer looks at the images that were flagged and doesn’t see CSAM, they don’t report it.
If they just hash photos, and if they only use the NCMEC database and if they only look at photos being uploaded and if the government has no access, then it is not anything crazy. But by implementing the system they invite scope creep at every one of those ifs.
Why did Apple do it, though? To create precisely this kind of cognitive dissonance. It's not Apple that's observing you. It's the government. Apple has basically created an API for government surveillance and can now say "see, we don't collect the data. We don't even know what big brother wants to know about you."
Apple just side-stepped the question on how to reconcile trust in propietary software and surveillance requests from the government. And everyone who tells you that it's not guaranteed that this feature will be extended to texts, spoken language, and other contents, is not worth talking to.
We now live in a world, where our personal computers come with "AI" that detects illegal use of those computers and automatically turns us in to the authorities. This general pattern won't disappear, ever. In a couple of years you will have to carefully consider every word you say in a car, every movie you watch, every website you visit, every program you write, because some neural net will use it to create a fingerprint that might be classified as illegal behavior.