I don't belive tech has an over weighted responsibility to solve society's problems, and in fact it's generally better if we don't try and pretend more tech is the answer.
Advocating for more money and more prioritization for this area of law enforcement is still the way to go if it's a priority area. Policing seems to be drifting towards "mall cop" work, giving easy fines, enabled by lazy electronic surveillance casting a wide net. Let's put resources towards actual detective work.
We deceive ourselves honestly by pretending like we have not created new realities which are problematic at scale. We have. They are plentiful. And if people aren’t willing that we walk back tech to reduce the problems and people aren’t willing to accept technical solutions which are invasive then what are we to do? Are we just to accept a new world with all these problems stemming from unintended consequences of tech?
Enabling people to talk to each other without all their communication being wiretapped and archived forever is not one of those, I would say.
Uber and AirBnB are just using very-widely-available technology—that some taxi services and hotels are also using!—and claiming that they're completely different when the main difference is that they're just ignoring the laws and regulations around their industries.
Cryptocurrencies are using a tech innovation as a front for what's 99.9999% a financial "innovation"...which is really just a Ponzi scheme and/or related scams in sheep's clothing.
LLMs are genuinely a tech innovation, but the primary problem they bring to the fore is really a conversation we've needed to have for a while about copying in the digital age. The signs have been there for some time that such a shift was coming; the only question was exactly when.
In none of these cases is technology actually doing anything "messed up". Companies that denote themselves as being "in the tech industry" do bad things all the time, but blaming the technology for the corporate (and otherwise human) malfeasance is very unhelpful. In particular, trying to limit technological progress, or ban widely useful technological innovations because a small minority of people use them for ill, is horrifically counterproductive.
Enforce the laws we have better, be more willing to turn the screws on people even if they have lots of money, and where necessary put new regulations on human and corporate behavior in place (eg, requiring informed consent to have works you created included in the training set of an LLM or similar model).
You haven't explained the problem 'tech' has created, I'm confused as to what your point is?
CSAM isn't a problem caused by 'tech' unless you're going back to the invention of the camera, and I think that toothpaste is well out of the tube.
Additionally, and this is where a whole of arguments about this go wrong, the important part: the actual literal abuse, is human to human. There is no technology involved whatsoever.
Technological involvement may be an escalation of offense, but it's vanishingly secondary.
Apple's proposed solution would have theoretically only reported cases that were much more than likely to be already known instances of CSAM (i.e. not pictures of your kids), and if nothing else is reported, can we say that they were really surveilled? In some very strict sense, yes, but in terms of outcomes, no.
Also, you can not guarantee that apple/google will use only known instances of csam, what if, govt orders them/google to scan for other type of content under the hood, like documents or god knows what else bc govt want's to screw that person (for the sake of example let's suppose the targeted person is some journalist that discovered shady stuff and govt wants to put em in prison), bc you know, you don't have access to either algorithms and csam scan list that they are using, system could be abused and usually could means 'sometime' it will
I agree that the basic idea of scanning on device for CSAM has a lot of issues and should not be implemented. What I think was missing from the discourse was an actual look at what Apple were suggesting, in terms of technical specifics, and why that would be well designed to not suffer from these problems.
https://www.texastribune.org/2022/07/26/texas-foster-care-ch...
So how about we implement mass-surveillance by giving victims a good reason to report crimes? Starting with not heavily punishing victims that do come forward. Make the foster care system actually able to raise kids reasonably.
Because, frankly, if we don't do it this way, what's the point? Why would we do anything about abuse if we don't fix this FIRST? Are we really going to catch sexual abuse, then put the kids into a state-administered system ... where they're sexually, and physically, and mentally, and financially abused?
WHY would you do that? Obviously that doesn't protect children, it only hides abuse, it protects perpetrators in trade for allowing society to pretend the problem is smaller than it is.
In terms of outcomes, almost nobody is actually surveilled, as the overall effect is the same as no data having been collected on them in the first place.
That said, I am personally more comfortable with my country's intelligence agencies hoovering up all my online activity than I am with the likes of Apple. The former is much more accountable than the latter.
What if you were a candidate for political office, pushing opinions that angered large swaths of the Intelligence Comminity?
The "minuscule fraction" of content is not surfaced by some random roll of the dice - it's the definitionally most interesting content, in the sense that some human went specifically looking for it in the heap of content caught in the dragnet. And it only needs to be interesting to at least one person with the clearance to search for it.
Maybe that means it's a video of a child being abused, and some morally upstanding federal officer is searching for it because anyone possessing it is ethically and legally culpable for the abuse of that child... Or maybe it's a PDF containing evidence of FBI kidnapping and torturing innocent civilians, and some morally corrupt federal officer is searching for it because anyone possessing it is a liability who needs to be silenced... Or maybe it's a JSON file containing the GPS locations of an individual for the past year, and some emotionally scorned federal contractor is searching for it because that individual is their ex-spouse who's moved onto a new partner.
Are you really prepared to put your faith in the trustworthiness and moral clarity of the population of 100k+ people with federal security clearances?
The scenarios you invented sound very far-fetched to me, if these did happen I very much doubt the perpetrator would be able to get away with it.
https://www.eff.org/deeplinks/2023/04/internal-documents-sho...
> At least a dozen U.S. National Security Agency employees have been caught using secret government surveillance tools to spy on the emails or phone calls of their current or former spouses and lovers in the past decade, according to the intelligence agency’s internal watchdog.
https://www.reuters.com/article/us-usa-surveillance-watchdog...
If illegal data (CP) is being transferred on the net, wiretapping that traffic and bringing hits to the attention of a human seems like a proportional response.
(Yes, I know, it's not going to be 100% effective, encryption etc, but neither is actual detective work.)
Congrats, you've got a religion.
If you want to get it amended, then by all means, make a case for why it should be amended.
In the meantime you wanted to know why mass surveillance isn’t an option. The answer “because it’s against the law” is a simple, good answer.
If you want to know why we decided as a nation to make that such a fundamental law that it is in our constitution, you could do worse than reading about what prompted the writing of the Bill of Rights.
I agree with a lot of the original reasoning.
While often true, at all times there have also been morally wrong laws, so it would not be unreasonable to counter that being written into law on itself means nothing. So you should always be prepared to pull out and defend the reasoning behind a law, which you also hinted at in your following sentences.
For this I need proof.
But it’s just begging the question to say there’s immense benefit to them searching US citizens’ communications without a reason.
That’s the whole question.
Show me why we should change the constitution which guarantees us freedom from this sort of government oppression.
Some of this will include data of British citizens, but the thing is, we have a significant home-grown terrorism problem and serious organised criminal gang activity, happening within the country. If intelligence analysts need to look at, for example, which phone number contacted which other phone number on a specific date in the recent past, there's no other way to do this other than bulk collect all phone call metadata from the various telecom operators, and store it ready for searching.
The vast majority of that data will never be seen by human eyes, only indexed and searched by automated systems. All my phone calls and internet activity will be in there somewhere, I'm sure, but I don't consider that in itself to be government oppression. Only if it's used for oppressive purposes, would it become oppressive.
I think we’re well beyond that point now. Whether or not encryption is allowed or not and however private you believe your virtual life to be, in the physical world surveillance is the norm. Your physical location, biometric information, and relationships can and will be monitored and recorded.
Absolutely.
> It is also bad for the fabric of society at large, in the sense that if we don't clearly take a stand against abhorrent behaviour then we are in some sense condoning it.
Much less clear. There's always been the argument: does it provide an outlet with no _new_ (emphasis on "new" so people don't skim that word) victims, or does it encourage people to act out their desires for real?
I don't have any answer to this question; but the answer matters. I do have a guess, which is "both at the same time in different people", because humans don't have one-size-fits-all responses.
Even beyond photographs, this was already a question with drawings; now we also have AI, creating new problems with both deepfakes of real people and ex-nihilo (victimless?) images.
> Does the tech industry have any alternate solutions that could functionally mitigate this abuse?
Yes.
We can build it into the display devices, or use a variation of Van Eck phreaking to the same effect.
We can modify WiFi to act as wall-penetrating radar with the capacity to infer pose, heart rate, and breathing of multiple people nearby even if they're next door, so that if they act out their desires beyond the screen, they'll be caught immediately.
We can put CCTV cameras everywhere, watch remotely what's on the screens, and also through a combination of eye tracking and (infrared or just noticing a change in geometry) who is aroused while looking at a forbidden subject and require such people to be on suppressants.
Note however that I have not said which acts or images: this is because the options are symmetrical under replacement for every other act and image, including (depending on the option) non-sexual ones.
There are places in the world where being gay has the death penalty. And if I remember my internet meme history right, whichever state Mr Hands was in, accidentally decriminalised his sex when the Federal courts decided states couldn't outlaw being gay because they had only one word in law for everything "unnatural".
Much like other 'tough on crime' measures (of which destroying E2EE is one) the real problems need to be solved not at the point of consumption (drugs, guns, gangs, cartels) but at the root causes. Getting rid of E2EE just opens the avenue for the abuse of us by the government but in no way guarantees we'll meaningfully make children safer.
And no, we are not 'condoning' it when we declare E2EE an overall good thing. Real life is about tradeoffs not absolutes, and the tradeoff here is protection from the government for potentially billions vs. maybe arresting a few thousand more real criminals. This is a standard utilitarian tradeoff that 'condones' nothing.
Governments are not good at this type of thing. It requires careful analysis, planning and actual decisions.
But slap on a regulation and require private companies to do the hard work for you - now we are talking!
Agreed. I'm sorry if I worded things in a way that caused you to see an implication which was not intended. To be clear: E2EE is a good thing. Championing E2EE is not equivalent to condoning CSAM.
What I did say is that in failing to try and provide any meaningful solutions to this unintended consequence of E2EE, the industry is effectively condoning the problem.
> This is a standard utilitarian tradeoff
If that's the best we can do, I'm very disappointed. That position says that to achieve privacy, I must tolerate CSAM. I want both privacy and for us not to tolerate CSAM. I don't know what the solution is, but that is what I wish the industry were aiming for. At the moment, the industry seems to be aiming for nothing but a shrug of the shoulders.
Not true, you can have privacy and at the not tolerate child pornography, those are two perfectly compatible positions and arguably the current state. What you can not have - by definition - is privacy on the one hand and on the other hand no privacy in order to look for child pornography. You can still fight child pornography in any other way, but when it comes to privacy, you have to make a choice - give people their privacy or look through their stuff for illegal content, you can not have both. If you have enough evidence, a judge might even grant law enforcement the permission for privacy violating measures, it should just not be the default position that your privacy gets violated.
Ah, here's where we disagree.
Let's take a practical example: Why can't the providers perform metadata analysis on traffic externals, and identify significantly-sized closed groups of participants? The providers could then pass those clusters over to law enforcement to take a closer look. Most of the clusters would probably be nothing, or at least not meet the threshold to get a warrant for further investigation. Some of the clusters might have significant participants who are already on law enforcement's radar, but below the line for investigation.
This would be a definite infringement on overall privacy. It would also be a potential step towards improving the CSAM situation. It would not require client-side scanning.
I don't know how palatable this suggestion is. My point is, compromises are possible. This is just one example -- I'm sure there are others. The technology industry is not, I believe, currently actively seeking these compromises.
Failure to recognize this perpetuates the fundamental inconsistency.
[1] https://en.wikipedia.org/wiki/Relationship_between_child_por...
I want to have more money than Elon Musk..... sometimes life is not fair and we can not always get what we want...
Any "backdoor" or "frontdoor" in encryption is a total failure of encryption. That is a immutable truth, more fixed in reality than the speed of light is in physics.
If you can't even propose a hypothetical science fiction-esque way of achieving this, much less one that might actually be implementable now or in the near future (like, 5-10 years), you shouldn't get to have that opinion.
In my opinion CSAM is a symptom, not a cause.
It's difficult to "stumble across" that kind of material unless you're already actively looking for it, which means some amount of "damage" is already done.
I also highly doubt that someone with no proclivities in that direction would 'turn' as a result of stumbling across CSAM. I'd guess they'd go the other way and be increasingly horrified by it.
It's entirely likely that borderline cases (15-17 years old) are seen by millions of people without them realizing it. Pornhub is a popular "mainstream" porn website that has issues with CSAM and routinely removes it when found. It's entirely possible for "normal" consumers of pornographic material to stumble into CSAM in places like that unknowingly. When you're talking about obviously prepubescent children, I'm in full agreement that it's almost always restricted to folks who seek it out explicitly.
Should we disallow encrypted traffic between machines because that encrypted traffic could be CSAM?
Saying "X is bad so we should be rid of it" requires a method to get rid of it.
Obviously giving up privacy and allowing more surveillance can reduce something but that isn't necessarily the right choice.
Like what do you do if they start sharing in files the user encrypts?
This is a very big assumption. Sexual abuse of minors has existed long before the internet, and long before photography. The notion that less availability of CSAM leads to less real-world abuse is not at all clear.
> If that's the best we can do, I'm very disappointed. That position says that to achieve privacy, I must tolerate CSAM. I want both privacy and for us not to tolerate CSAM. I don't know what the solution is, but that is what I wish the industry were aiming for. At the moment, the industry seems to be aiming for nothing but a shrug of the shoulders.
As other commenters have pointed out, the solution is to prevent children from being abused in the first place. Have robust systems in place to address abuse, and give kids effective education and somewhere to speak out if it happens to them or someone they know.
Yes, and you can tell because the proposed solutions attack privacy when alternative solutions exist.
For example, simply deleting CSAM material from devices locally without involving any other parties could have achieved the goals without privacy violations.
It makes me somewhat uncomfortable to argue for not involving other parties (like the police) in cases where real CSAM is found on someone’s device. Same as most people, I think that CSAM is morally reprehensible and really harmful to society. But just deleting it en-masse would have been an effective and privacy-respecting solution.
I think it’s important to see nuance even in things we don’t like to think about. Not everything that has a price tag has a price. We were told we needed to give up privacy, but that wasn’t necessary to take CSAM out of circulation.
...Understand I don't see it that way and applaud you for your way of thinking. I too have had to wrestle with the very uncomfortable "bed fellows" as it were that adhering to consistent application of principles inevitably results in.
The fact remains though that in a large swathe of the population, the ripping off and sacrifice of personal privacy is considered a small price to pay to inflict harm on that subpopulation. My issue comes in in that once you make the exception for one subpopulation, the slope is set.
Though even your "just delete it" has dystopian ramifications. Imagine that were implemented like Tianenmen Square? Recordings that are not blessed? You're still leaving in somebody's hands essentially executive control over what information can be allowed to exist, which is an unconscionably powerful lever to build.
This is one of those rare circumstances where "do nothing and clean up the mess" may be the most wise course of action.
Blah blah blah, the same old argument given by the "think of the children" people.
There are many ways to counter that old chestnut, but really, we only need to remember the most basic fundamental facts:
1) Encryption is mathematics 2) Criminals are criminals
Can you ban mathematics ? No. Can you stop criminals being criminals ? No.
So, let's imagine you are able to successfully backdoor E2EE globally, on all devices and all platforms.
Sure, the "think of the children" people will rejoice and start singing "Hallelujah". And the governments will rub their hands with glee with all the new data they have access to.
But the criminals ? Do you honestly think they'll think "oh no, game over" ?
No of course not. They'll pay some cryptographer in need of some money to develop a new E2EE tool and carry on. Business as usual.
> No of course not. They'll pay some cryptographer in need of some money to develop a new E2EE tool and carry on. Business as usual.
I used to think this, I changed my mind: just as it's difficult to do security correctly even when it's a legal requirement, only the most competent criminal organisations will do this correctly.
Unfortunately, the other issue:
> And the governments will rub their hands with glee with all the new data they have access to.
Is 100% still the case, and almost impossible to get anyone to care about.
Assuming no new zero-days, and that they were even doing best practices in the first place. How many legit companies do best practices?
That's the point, bad actors can circumvent the system if they feel threatened, but system can be exploited by gov/companies once rolled out globally to target any user, so we get something that may be not that effective against bad actors but poses great risk to be misused by gov/company in their own interests without users knowing. I've seen how an authoritarian gov in my country is targeting ppl bc they are uncomfortable for the system and this algorithm opens another potential vector of attack.
> I've seen how an authoritarian gov in my country is targeting ppl bc they are uncomfortable for the system and this algorithm opens another potential vector of attack.
As per my last sentence in my initial comment in this chain:
--
> And the governments will rub their hands with glee with all the new data they have access to.
Is 100% still the case, and almost impossible to get anyone to care about.
All it takes is for one criminal to write a one-page guide to using GPG and circulate it to the group ....
I know I mentioned paying a cryptographer earlier, but in reality downloading and using GPG is a crude and effective way of defeating an E2EE backdoor.
Are the GPG devs going to backdoor GPG to satisfy governments ? Probably not.
If cybersecurity was that easy, we wouldn't have so many examples of businesses getting it wrong.
Just because everyone here can follow instructions like that, doesn't make it common knowledge for anyone else.
I can only partially agree with this point. Businesses getting cybersecurity wrong has almost no material and significant consequences. At best, they get a tiny slap on the wrist or asked to answer some questions. Nobody in said businesses goes to jail for it or personally pays any fines. Compare that to criminals who have a lot more to lose if they get caught — jail time, fines they have to pay, not having freedom for quite sometime, life not being the same after they’ve served their sentence, and more. Businesses have it extremely easy compared to this. No wonder cybersecurity is so poor among all businesses, including very large ones (like Microsoft, as a recent example).
Fear of these is the reason for the (maliciously compliant) GDPR popups, and that despite discussion about extra-territoriality and relativity limited capacity-to-websites ratio.
The law and threats of punishment are clearly not hugely significant to anyone involved in the specific topic of this thread regardless; in the UK at least, it's the kind of thing where if someone is lynched for it, the vigilantes have to be extremely stupid (like attacking a paediatrician because they can't tell the difference, which happened) to not get public sympathy.
There are almost no consequences for anyone working at a business that gets it wrong.
The consequences for being a nonce are quite severe so the motivation to get it right will be quite high.
GPG is infamous for being difficult to use correctly and for an antiquated design (IIRC forward secrecy is impossible?). And assuming E2EE backdoor actually exists, the gov is likely to be able to get at your key.
>Are the GPG devs going to backdoor GPG to satisfy governments?
No, but most users are unlikely to verify their GPG build is the right build.
[Citation needed]
This mindset—that assigns people into immutable categories, "criminal" and "not criminal"—is actually one of the biggest things that needs to change.
We absolutely can stop criminals from being criminals. We just can't do so by pointing at them and saying "Stop! Bad!" We have to change the incentives, remove the reasons they became criminal in the first place (usually poverty), and make it easier, safer, and more acceptable to go from being a criminal to being a not-criminal again.
Yet, the only justification around for mass surveillance for camera pictures, the important societal matter to rest literally on "the other side of the coin", is "some minority of people could be sharing naked photos of young members of society for highly unethical acts of viewing".
wtf.
Oh, please. As if we couldn't just compare the hashes of the pictures people are storing against a CSAM database of hashes that gets regularly updated
When this was proposed people would respond "But they could just mirror the pictures or cut a pixel off!"
Who cares? You got that picture from some place in the dark web, and eventually someone will stumble upon it and add it to the database. Unless the person individually edits the pictures as they store them, that makes it so that you're never sure if your hashes will posthumously start matching against the DB.
People who wank off to CSAM have a user behavior similar to any other porn user, they don't store 1 picture, they store dozens, and just adding that step makes them likely to trip up, or straight up just use another service altogether
"What if there's a collision?" I don't know, go one step further with hashing a specific part of the file and see if it still matches?
This whole thing felt like an overblown fearmongering campaign from "freedom ain't free" individualists. I've never seen anything wrong with content hosters using a simple hash against you like this.
But even if there was an md5 hash collision back when md5 was the only one hash use, it still doesn't matter because upon viewing the image that matched, if it's not csam, it doesn't matter. Having said that, the chance of dozens of images matching hashes known to be associated to csam is also so unlikely as to be unthinkable. Where there is smoke, there is fire.
And further, a hash alone is meaningless, since in court there must be a presentation of evidence. If the image that set off the csam alarm by hash collision is say, an automobile, there is no case to be had. So all this talk about hash issues is absolutely moot.
Source: I have worked as an expert witness and presented for cases involving csam (back when we called it Child Pornography, because the CSAM moniker hadn't come about yet), so the requirements are well known to me.
Having said all that, I am an EFF member, and I prefer cryptography to work, and spying on users to be illegal.
That's why people were initially all up in arms about
But they didn't need to do that.
A regular hash where "just changing one pixel would be enough to evade the scanner" is already good enough
So when you say "a hash alone is meaningless, since in court there must be a presentation of evidence", you'd just present the image to court.
The hash is the trigger to call the authorities they handle the rest
And with a userbase the size of apple and people as pissy as reddit, you want to completely exclude a possibility of collisions or you'll get a repeat of the scenario we got in 2021
https://www.nbcphiladelphia.com/news/local/child-porn-victim...
Federal and state police, some of the best funded, equipped and trained police in the world are so inundated with cases that they are forced to limit their investigations to just toddlers and babies. What use is it to add more and more cases to a mountain of uninvestigated crimes? Whats needed is more police clearing the existing caseload.
That's an unbelievably depressingly low bar for society.
How many times will actual abusers be allowed to go free while your own family is victimized by the authorities and what ratio do you find acceptable?
I think that companies might need to enable some kind of mechanism for offline investigation of the devices though. The CSAM is a real problem, there are real predators out there, the only risks isn't CSAM and law enforcement does really need to have a way to investigate devices. Previously, my proposal was the ability to force the device scan the user content for fingerprints of the suspected content but only with physical access. Physical access enforces the law enforcement to actually have a real and official investigation with strong enough reasons to spend resources and risk repercussions when done improperly.
However, the project of scanning all the user content for policing the users was one thing that irked and later relieved when Apple abandoned the project.
Apple's explanation is good and I agree with them but IMHO the more important aspects are:
1) Being able to trust your devices being on your side. That is, your device shouldn't be policing you and shouldn't be snitching on you. At this time you might think that the authorities who would have controlled yor device are on your side but don't forget that those authorities can change. Today the devices may be catching CSAM, some day the authorities can start demanding catching people opposing vaccines and an election or a revolution later they can start catching people who want to have an abortion or having premarital sexual relations or other non-kosher affairs.
2) Being free of the notion that you are always watched. If your device can choose to reveal your private thoughts or business, be it by mistake or by design, you can no longer have thoughts that are unaligned with the official ones. This is like the idea of a god who is always watching you but instead of a creator and angels you get C level businessmen and employees who go through your stuff when the devices triggers decryption of your data(by false positives or by true positives).
Anyway, policing everyone all the time must be an idea that is rejected by the free world, if the free world doesn't intent to be as free as Democratic People's Republic of Korea is democratic.
Indeed, they are correct. And they were also brought up when Apple announced that they would introduce this Orwellian system. Now they act like they just realized this.
Cops need to investigate the same way they always have, look for clues, go undercover, infiltrate, find where this stuff is actually being made, etc.
Scanning everyone's phones would make their jobs significantly easier, no doubt, but it simply isn't worth the cost to us as a society and there is simply no good counter-argument to that.
"Apple" wasn't scanning your phone, neither was there a "backdoor".
If you would've had iCloud upload enabled (you'd be uploading all your photos to Apple's server, a place where they could scan ALL of your media anyway), the phone would've downloaded a set of hashes of KNOWN and HUMAN VERIFIED photos and videos of sexual abuse material. [1]
After THREE matches of known and checked CSAM, a check done 100% on-device with zero data moving anywhere, a "reduced-quality copy" would've been sent to a human for verification. If it was someone sending you hashbombs of intentional false matches or an innocuous pic that matched because some mathematical anomaly, the actual human would notice this instantly and no action would've been taken.
...but I still think I was the only HNer who actually read Apple's spec and just didn't go with Twitter hot-takes, so I'm fighting windmills over here.
Yes, there is always the risk that an authoritarian government could force Apple to insert checks for stuff other than CSAM to the downloaded database. But the exact same risk exists when you upload stuff to the cloud anyway and on an even bigger scale. (see point above about local checks not being enabled unless iCloud sync is enabled)
[1] It wasn't an SHA-1 hash where changing a single bit in the source would make the hash invalid, the people doing that were actually competent.
Yes, you're right. But I've seen calls for phones to scan all content before being uploaded or encrypted, and it often feels, at least in some countries, that could still plausible happen. I suppose that's what I had in mind when I wrote my comment.
> But the exact same risk exists when you upload stuff to the cloud anyway and on an even bigger scale.
There's a difference with them actively doing it and announcing they are doing it, vs the possibility they are doing it silently without consent.
It's a cost of doing business pretty much, you need to give the authorites access to customers data or they can go "but think of the children, there might be child abuse material in there!" and that's really damn hard to argue against.
Thus: checking stuff on-device and keeping the cloud locked so that not even the hoster can access it nor can they create a backdoor.
At least some cloud providers require a warrant before doing so.
But the point is that if the data is fully encrypted, no warrant will help against pure mathematics. A cloud provider cannot give something they have no access to.
Right, but your post kind of made it seem like they didn't. And with the CSASM stuff, they didn't need anything to match hashes. So what happens when that gets expanded to other types of content that they still don't need a warrant for? Like torrent files?
> But the point is that if the data is fully encrypted, no warrant will help against pure mathematics. A cloud provider cannot give something they have no access to.
Agreed, but most don't use encryption, and should still have privacy rights for their data.
The point of the system would've been that EVERYONE'S content would've been encrypted in the cloud without them needing to do anything.
also: If it was someone sending you hashbombs of intentional false matches or an innocuous pic that matched because some mathematical anomaly, the actual human would notice this instantly and no action would've been taken. - if someone is doing this, imagine the scale- thousands of pics that should be human-evaluated, scaled to thousands of people, it'll be just plain ignored, meaning system loses it's purpose. also, you say that it'll be enabled only if icloud bck is enabled, but it's not guaranteed, this assumption can later change... and it doesn't make sense, for me your 2 statements contradict themselves:
- if apple can scan your photos in icloud AND for this feature to be enabled, you must enable icloud, why they should send hashes to you? they can scan the photos anyway in icloud, since all your photos are backed up there. Unless... they can't scan photos in icloud since these are encrypted, meaning scanning can be done only locally before photos are sent, meaning icloud enabling is not mandatory and it could work without it.
Either way the csam scanning is imo pointless, on one hand bc of privacy reasons(and we've seen that if state is able to use a backdoor, it'll use it when needed) and on the other hand, because of generative algorithms: photos can be manipulated to trigger csam even if human eye can see another thing (aka hashbomb) OR a sick/ill intentioned person can generate a legit csam like photo just by using target ppl's face(or description of their face), in this case I don't even know if they are breaking the law or not, since the image is totally generated but is looking totally illegal
"Looking illegal" isn't enough to trigger a CSAM check in this case. It's perfectly normal to take pictures of your own kids without clothes in most of Europe for example. Nothing illegal.
That's why the checks would've been explicitly done on known and confirmed CSAM images. It wasn't some kind of check_if_penis() -algorigthm, or one of the shitty ones that trigger if there's too much (white) skin colour in an image.
Outsourcing work for this(afaik) isn't possible since it's private data, not public and only specific organisations can have full access to potential triggers
But in the end it also doesn't matter because there are other problems too, like how to make the final list easily checkable so that we are sure governments/ companies do not alter the list to target specific ppl/groups for their own interest. Or how algorithm isn't modified under the hood to check not just images but also text/files
Nobody read the bit about an actual human verifying results before any law enforcement would be called in.
And outsourcing checking is a huge industry even today[0]. How do you think the huge social media companies keep CSAM, gore etc out of their systems? They're not using Pied Piper's hotdog or not algorithm, that's for sure.
[0] https://www.theverge.com/2019/2/25/18229714/cognizant-facebo...
A snippet from the article:
> The video depicts a man being murdered. Someone is stabbing him, dozens of times, while he screams and begs for his life. Chloe’s job is to tell the room whether this post should be removed.
Nobody read the bit about an actual human verifying results before any law enforcement would be called in. - again, if system can be gamed, human check is useless. Imagine someone will generate 100k false positives, multiply this by 1k ppl and imagine among them there is a real bad person with 10 real csam images and another person with fully generated csam images (like generated porn is a thing now so generating csam like images is possible). How do you think gov will human evaluate 100kk pictures that are triggering the system? Because if they can't, system is useless for this usecase but is still useful for potential gov oppression or company ad targeting
You can generate a billion false positives and it still won't do anything. You need to get them to people's iCloud photo libraries first. Each library needs to have multiple false positive images before triggering a human check. They intentionally didn't tell the exact number needed, but it's not two.
If you have a way to get enough people to get fake CSAM material on their phones to overwhelm the human checkers, why would you waste in on something stupid like that? Just by forcibly inserting advertisements to peoples Photo Libraries would make you a billionaire. Not an ethical one, but still rich.
Oh, and just for reference. FB gets 350 million photos uploaded every day and they keep it moderated just fine. The amount of people you'd need colluding with you to overwhelm the system Apple had designed would be staggering.
And then you've achieved what? Make it possible to share child pornography because you broke the system? Yay, victory?
The local scanning would've been enabled only if you would've uploaded the photos to iCloud anyway.
To complete your crappy analogy: You had already agreed to send the information of everything contained in your apartment to a company, and now you're getting into a hissy-fit when they are sending a robot visit your house with a list of known child pornography photos and checking if you have any?
There weren't any "hard decisions" it wasn't a "this has a naked person in it" -scanner. It wasn't an "abuse checked" it was specifically created to find known CP.
It checked your photos against a NeuralHash of KNOWN child pornography. So the only chance your sauna photos would've been flagged if they were included in a set of CP photos distributed widely enough so that they're added to the CSAM database.
And nobody claimed the system would magically cure the world of grooming, where did you get that idea from? Although the current filters in iMessage might help if the abuser is stupid enough to use that for grooming.
Yes, i know they scan all photos already. But to scan MY photos and hold me accountable, that is something i am not okay with. Yes, its only icloud, for now, but i dont see the point of using some 3rd party cloud, i used to, but in apple device it would be suboptimal for my daily use etc etc.
My point was of the original case of protecting children. That is what the advocate groups want. That is what I want. But I just draw the line there what i feel is my personal space, that you don't scan my photos or files, and that is not the answer. Since there are these few behemoth FAANG companies ruling the ecosystems and impacting our lives, the battle of privacy needs to be fought right there. So, for me that would be the line as I see the progression of that path is much worse. That is where I raise my hand and say, I'm out, what next? No smartphone? Maybe. :) Peace.
They can find those materials the same way abusers work their way into communities to have access to them in the first place.
The increased scale only means they need more people working on it.
That aside, we could also, at least in the US, stop taking such a ridiculous stance against drugs and instead prioritize finding makers of CSASM materials.
The people are there, they are just not being utilized effectively.
It hasn't particularly changed production, which is where the actual abuse happens. There are still actual people abusing and filming actual children, and those can be found by the police by the same old-fashioned methods they've always had available. (Plus many new ones that don't violate everyone's civil liberties or destroy the security of every networked device.)
> Does the tech industry have any alternate solutions that could functionally mitigate this abuse? Does the industry feel that it has any responsibility at all to do so? Or do we all just shout "yay, individual freedom wins again!" and forget about the actual problem that this (misguided) initiative was originally aimed at?
The issue at large here is not the tech industry, but law enforcement agencies and the correctional system. Law enforcement has proven time and time again themselves that the most effective way to apprehend large criminal networks in this area is by undercover investigation.
So no, I don't think it is the tech industries role to play the extended arm of some ill conceived surveillance state. Because Apple is right: This is a slippery slope and anyone that doesn't think malicious political actors will use this as a foot in the door to argue for more invasive surveillance measures, using this exact pre-filtering technology are just naive idiots, in my oppinion.
Is being against child sexual abuse really an unusual opinion in the tech industry? Have we really all gone so far along the Heinlein/Rand road that any mention of a real negative outcome gets immediately dismissed with the empathy-free thought-terminating-cliche "think of the children?"
Nobody said that. You are being manipulative an trying to make it look that people who disagree with you are somehow pro-child abuse.
In saying "Does the tech industry have any alternate solutions that could functionally mitigate this abuse?" you are trying to pain a picture in which child abuse is somehow "tech industry" fault.
You are also trying to paint a complete Panopticon in which every interpersonal communication is subjected to surveillance by the state as somehow the default, that end to end encrypted electronic communication is changing - while the truth is that personal communication was private for hundreds of years, because it was impossible for the state to listen in on everything.
> You are being manipulative an trying to make it look that people who disagree with you are somehow pro-child abuse.
I am not doing that. You described my position as an "alien morality", to which another poster seemed to agree. I was responding to that by clarifying the actual moral point I was making. For the avoidance of doubt, I am not arguing that you are pro-child abuse.
> In saying "Does the tech industry have any alternate solutions that could functionally mitigate this abuse?" you are trying to pain a picture in which child abuse is somehow "tech industry" fault.
Yes, this is basically a correct understanding of my position. I am stating that the problem has been massively exacerbated by the adoption of E2EE by the tech industry, and that the tech industry therefore has a moral responsibility to deal with the unintended consequences of its own action.
> the truth is that personal communication was private for hundreds of years, because it was impossible for the state to listen in on everything.
The truth is that in pre-internet times, networks of pedophiles could be dismantled by the state, partly through surveillance -- see https://en.wikipedia.org/wiki/Paedophile_Information_Exchang... for a supporting example.
I understand that most information on how the state fights organised crime will be classfied, but if there is any publicly available evidence for this claim that you can share, I would be interested in reading it (and I hope others on this thread would be too). I'm not saying I doubt you - you give the impression you know what you're talking about - please take this in the spirit it's intended, as one of my former supervisors once said "In academia, asking for citations/references is an expression of interest, not of doubt".
I'm not part of any state, and I don't have access to any special knowledge that you can't find on the internet.
I'm also not aware of any study that provides the very direct link you're asking for. Because of the nature of E2EE, I don't know if it would be possible to produce one. What I can do is link to evidence such as https://www.weprotect.org/global-threat-assessment-21/#repor..., which has (to me) some fairly compelling data showing that the magnitude of the problem is increasing.
The moral part is whether and how much society / the state / the tech industry should invest in combating it, and how the advantages and disadvantages of mandating government access to E2E encrypted communications or people's cloud storage weigh up.
For what it's worth, my own position is that the state should do more about it, and should in principle have more resources allocated to do so. I would support higher taxes in exchange for more police (and better trained police), who could do more about many kinds of crime including child abuse. I wouldn't mind more resources being allocated to policing specifically for fighting child abuse, too. But I could think of a lot of other places besides legislating access to people's messenger apps where such resources could be invested.
I'm still undecided on whether legally mandated backdoors in E2E encrypted storage and communications would be _effective_ in fighting child abuse, which is a question I would need more technical knowledge on before I could take an informed position (I know a fair bit about cryptography but less about how organised crime operates). If it turns out that this would be an ineffective measure (maybe criminals fall back on other means of communication such as TOR relays) then it would be hard to justify such a measure morally, especially as it could have a lot of disadvantages in other areas.
https://news.ycombinator.com/newsguidelines.html
Edit: we've had to warn you about this kind of thing many times before:
https://news.ycombinator.com/item?id=33476972 (Nov 2022)
https://news.ycombinator.com/item?id=30612118 (March 2022)
https://news.ycombinator.com/item?id=26857565 (April 2021)
https://news.ycombinator.com/item?id=21678059 (Dec 2019)
https://news.ycombinator.com/item?id=18550058 (Nov 2018)
We eventually have to ban accounts that won't stop breaking the rules. I don't want to ban you, so if you'd please review https://news.ycombinator.com/newsguidelines.html and stick to them, we'd appreciate it.
I'm all for privacy, but those who put it above all else are already likely not using Apple devices because of the lack of control. I feel like for Apple's target market the implementation was reasonable.
I think Apple backed down on it because of the vocal minority of privacy zealots (for want of a better term) decided it wasn't the right set of trade-offs for them. Given Apple's aim to be a leader in privacy they had to appease this group. I think that community provides a lot of value and oversight, and I broadly agree with their views, but in this case it feels like we lost a big win on the fight against CSAM in order to gain minor, theoretical benefits for user privacy.
Idk, maybe i'm overreacting, but I've seen what a repressive gov can do, and with such an instrument it's frightening what surveillance vectors can be opened
I'd suggest there's a lot the not-tech industry could do to stop condoning abhorrent behavior that stops short of installing scanners on billions of computing devices. It's become a bit of a trope at this point, but it's bizarre to see a guy who is/was spokesman for a "minor attracted persons" (i.e. pedos) advocacy group getting published negatively reviewing the controversial new sex trafficking movie ... in Bloomberg:
https://www.bloomberg.com/opinion/articles/2023-07-15/qanon-...
For some background:
https://www.opindia.com/2021/08/meet-noah-berlatsky-prostasi...
His 501(c)(3) non-profit also advocates in favor of pedophilic dolls and has a "No Children Harmed certification seal" program for pedophilic dolls/etc:
https://prostasia.org/blog/dolls-prevent-child-sexual-abuse/
https://prostasia.org/no-children-harmed/
I'm not sure you can criminalize stuff like this, but it sets off my alarm bells when pedophile advocates are being published in mainstream news at the same time there's a moral panic around the need to scan everyone's hard drives. Is society actually trying to solve this problem, or is this more like renewing the Patriot Act to record every American's phone calls at the same time we're allied with al Qaeda offshoots in Syria? Interesting how terrorism has been the primary other argument for banning/backdooring all encryption.
----
As an aside I couldn't find ~anything about this group "Heat Initiative" Apple is responding to? Other than a TEDx talk by the founder a couple years ago which again seems very focused on "encrypted platforms" as the primary problem that needs solving: https://www.ted.com/talks/sarah_gardner_searching_for_a_chil...
Looking at lack of response from law enforcement in regards to the visitors of Epstein island, the UK's lack of response to grooming gangs, and cases in recent years with short sentences for child molesters, I doubt Apple finding CSA materials and reporting them to law enforcement would result in any meaningful action in the vast majority of cases.
That makes the choice easier, have your privacy violated for no effect or don't.
That is a bigger problem and it will take a long time to fix. So long that I suspect that anybody reading this is long dead, but its like the saying with planting trees.
There is no parallel to be drawn between better encryption and worse outcomes for kids. Should we also outlaw high-performance cars because these sometimes serve as effective getaway vehicles for criminals?
CSAM producers and consumers should be found and punished via old-fashioned methods. How was this done in the past? Did we just never catch any human traffickers / rapists? No, we had detectives who went around detecting and presumably kicking down doors.
To outlaw large sections of mathematics because of this is absurd. And from the amount of power it would give big governments / big businesses, the fabric of society doesn't stand a chance.
Yea, LE/IC clearly have gone too far in many modern tactics.
Yea, it's possible to build a surveillance/police state much more efficiently than ever before.
Yea, we should be vigilant against authoritarianism.
What if we change the last bit after the "because" to "these sometimes are used at unsafe speeds and, intentionally or not, kill people who are not in cars?"
Because, at least for me, the answer is an unambiguous yes.
I agree that privacy and security should be available to everyone. But we also shouldn't count on being able to find people who are doing vile things--to children or adults--because the person messed up their opsec. I think Apple is correct here but as an industry we have to be putting our brains to thinking about this. "To outlaw large sections of mathematics" is hyperbole because we use mathematics to do a lot of things, some useful and some not.
How is this different from how police find anybody else who commits a crime? Like if they're trying to solve a murder, they're looking for DNA, clues, etc... They're literally looking for where the person who committed the crime "messed up their opsec" to use your wording.
Governments and law enforcement agencies have access to more information than they've ever had before. They have more cameras, location data, tons of data compiled by data brokers. But it's not enough - of course they have to have this too.
On top of that, there's a long history now of governments buying zero day vulnerabilities or even technology from firms like NSO and guess what? It's not being used to catch pedophiles (cue shocked Pikachu face) but it is being used to target political dissidents.
This is so frustrating, because it feels like a siege on a city. Collectively, people have to fight against bad legislation in various countries constantly. But the other side only has to "win" once.
I'd say that "we" as an industry should be putting out brains to trying to figure out ways to make it even harder for these people to legislate encryption out of existence, than trying to find ways to appease geriatric lawmakers.
Recently invented encrypted chat rooms allow people to coordinate and transfer CSAM without any government official being able to infiltrate it. And just being able to freely discuss has been shown to make the problem worse as it facilitates knowledge transfer.
This is all completely different to in the past where this would have been done in person. So the argument that we should just do what we did in the past makes no sense. As technology advances we need to develop new techniques in order to keep up.
Any source for that statement ? My understanding is that child abuse - while still existing - is at an all time low in modern western societies. Children working in brothel used to be common, children getting abused by the church used to be a well know fact etc.
Overall a using children is harder than ever even if it has not completely disappeared.
Technology - which allows efficient transfer of information, testimonies etc - is instrumental to that evolution.
Fuck that.
The "old-fashioned methods" that they used in the past included intercepting communications of people that were suspected of crimes, such as by getting a warrant allowing them to force the person's phone company to record and turn over the person's calls, or by getting a warrant to intercept and inspect the contents of the person's mail at the post office.
> To outlaw large sections of mathematics because of this is absurd
No one has or is proposing outlawing large sections of mathematics, or even small sections of mathematics. The laws are outlawing some applications that make use of mathematics.
Calling that outlawing mathematics is as absurd as saying that building codes that won't let me use asbestos insulation in new construction are banning sections of thermodynamics. Or saying that laws that restrict how high I can fly a drone are banning large sections of aerodynamics.
Stop equivocating. You're banning the mathematics. The mechanism is literally the mechanical implementation of the mathematics.
>Calling that outlawing mathematics is as absurd as saying that building codes that won't let me use asbestos insulation in new construction are banning sections of thermodynamics.
...Except that's not even an analogous comparison? The asbestos is forbidden not because it's too good an insulator/foiler of thermodynamics, but because of it's danger to the health of everyone.
Trying to ban applications that use encryption is exactly banning asbestos because it's too good an insulator, and you're interested in seeing whatever is wrapped in it burn.
Yes. If consumer cars’ speed was capped to 100 mph it would affect a very small percentage of people and zero legitimate use.
The difference with encryption is that while it protects criminals from the police, it also protects legitimate users from criminals.
How many more times do we have to see this movie before people remember how it ends.
I don't really buy any "slippery slope" arguments for this stuff. Apple already can push any conceivable software it wants to all of its phones, so the slope is already as slippery as it can possibly be.
It just doesn't make sense to say "Apple shouldn't implement this minimal version of photo-scanning now even though I don't think it's bad, because that's a slippery slope for them to implement some future version of scanning that I do think is bad." They already have the capability to push any software to their phones at any time! They could just skip directly to the version you think is bad!
your source for proof of that?
>> in the sense that if we don't clearly take a stand against abhorrent behaviour then we are in some sense condoning it.
No. this narrative of "silence is violence" and "no action is support" etc is 100% wrong.
You started out great, but i can not get behind this type of thinking...
>>Does the tech industry have any alternate solutions that could functionally mitigate this abuse?
Why is it a "tech industry" problem
>>Or do we all just shout "yay, individual freedom wins again!"
For me the answer is simple... Yes individual freedom is more important that everything. I will never support curbing individual freedom on the alter of any kind proclaimed government solution to a social safety problem. Largely because I know enough about history to understand that not only will they no solve that social safety problem, many in government are probably participating in the problem and have the power to exempt themselves, while abusing the very tools and powers we give them to fight X, for completely unrelated purposes.
Very quickly any tool we would give them to fight CSAM would be used for Drug Enforcement, Terrorism, etc. It would not be long before the AI based phashes detect some old lady's Tomato plants as weed and we have an entire DEA paramilitary unit raiding her home...
Creating backdoors that allow encryption schemes to be subverted is _fundamentally_ going to cause harm on the internet, and eventually fail the weakest users/those that need privacy/security the most.
A mechanism that can subvert cryptographic protocols can be used by any party, including oppressive regimes, private entities etc. that have the resources/will/knowledge to use the backdoor etc. Backdoors harm both the trust on the web (which can have an impact on economic transactions among many others) and the people that need security/privacy the most. In the meantime, criminals will wise up and move their operations elsewhere where no backdoors exist.
We basically end up with a broken internet, we are putting people in harm's way and the criminals we are targeting are probably updating their OPSEC/MO not to rely on E2EE.
...regardless of whether Apple rolls out E2EE right? End to end encryption is available through a whole host of open-source tools, and should Apple deploy CSAM scanning the crooks will just migrate to a different chat tool.
You mean at rates greater than in the 1970s when it was legal in the US and there were a bunch of commercial publications creating and distributing it (prior to the "Protection of Children Against Sexual Exploitation Act" in 1977 (which targeted the production) and "Child Protection and Obscenity Enforcement Act" in 1988 (which targeted the distribution))? I find that doubtful.
> Does the industry feel that it has any responsibility at all to do so?
Why should it? Should every home builder and landlord install a camera in every bedroom to stream back to the government to be sure that no children are being molested? Why not? It's technologically possible today.
The fact that garbage people do garbage things to vulnerable people doesn't mean that the entire world must be commandeered to stop it.
Both of these arguments are absolutely, unambiguously, correct.
The other side of the coin is that criminals are using E2EE communication systems to share sexual abuse material in ways and at rates which they were not previously able to. This is, I argue, a bad thing. Is is bad for the individuals who are re-victimised on every share. It is also bad for the fabric of society at large, in the sense that if we don't clearly take a stand against abhorrent behaviour then we are in some sense condoning it.
Does the tech industry have any alternate solutions that could functionally mitigate this abuse? Does the industry feel that it has any responsibility at all to do so? Or do we all just shout "yay, individual freedom wins again!" and forget about the actual problem that this (misguided) initiative was originally aimed at?