News for hackers. Stuff that matters.
Re:Apple kills plans to scan for CSAM in iCloud by infotogivenm on Wednesday December 07, 2022 @07:24PM (#33902409)
My understanding was: it was only ever scanning things on the client that were uploaded to iCloud Photos, in the same exact way they are scanned server-side today but arguably a step more “transparently” for the user (at least if the user is a reverse engineer, heh).

What was crazy about this? The media outrage around the PR was incoherent to me at the time, and it seemed no one took any time to understand the present realities or details.

Re:Apple kills plans to scan for CSAM in iCloud by n8cpdx on Wednesday December 07, 2022 @08:32PM (#33902941)
Your apartment complex decides they don’t want to be party to anything illegal. Just in case, they set up a police precinct in the lobby. They set up hidden cameras in every room of your apartment, and if their AI model detects anything suspicious, they send the video to the detective. Because you aren’t doing anything illegal, you have nothing to worry about, right?

What’s crazy about this?

Another way of expressing the concern: does your iPhone work for you, with the help of Apple’s services, or does your iPhone work for Apple? Working for me means not having software designed to report me to the police for how I use my device. The hash database for now includes CSAM hashes; there’s no reason it couldn’t be extended to include similarly heinous material, like Winnie the Pooh memes, Hong Kong freedom posters, rainbow flags, hormone therapy instructions, or anything else offensive to the regime.

People that are comfortable with their device working for a company with the assistance of the user choose Android.

Re:Apple kills plans to scan for CSAM in iCloud by xionon on Wednesday December 07, 2022 @08:50PM (#33903073)
The apartment analogy doesn't quite work, as the scanning only happened with photos that were going to be uploaded to iCloud.

It wasn't like constantly monitoring your private residence, more like setting up a breathalyzer checkpoint on the way to the freeway.

Re:Apple kills plans to scan for CSAM in iCloud by magicjosh on Wednesday December 07, 2022 @09:50PM (#33903506)
Fixed:

Your apartment complex decides they don’t want to be party to anything illegal. Just in case, they set up a police precinct in the lobby. They set up hidden cameras in the lobby, the hallway leading to your apartment, and the balcony of your apartment. "All public spaces of course" the crowd consents. And if their AI model detects anything suspicious, they send the video to the detective. Because you aren’t doing anything illegal, you have nothing to worry about, right?

Re:Apple kills plans to scan for CSAM in iCloud by infotogivenm on Thursday December 08, 2022 @04:40AM (#33906107)
You’ve just described every photo upload service in America, although my understanding was Apple would use a list of hashes of known bad content, not an “AI” as google does.

Everyone scans for CSAM. I am not conjecturing on the ethics of scanning photos here, I am suggesting that moving from server- to client-side scanning had no effect on any of the things you are ranting about. Hence why I do not understand the outrage.

> Working for me means not having software designed to report me to the police for how I use my device.

Let me fix it: “how I use iCloud Photos, a hosted service on Apple’s servers”.

Re:Apple kills plans to scan for CSAM in iCloud by thewebcount on Wednesday December 07, 2022 @11:43PM (#33904254)
Isn’t this literally done in most office buildings in the US and probably a lot of apartment buildings already? There are CCTV cameras everywhere in the US (and other countries).
Re:Apple kills plans to scan for CSAM in iCloud by nyani on Wednesday December 07, 2022 @09:23PM (#33903300)
Closer to scanning your vehicle for meth on the way to the freeway. Rather fraught analog either way perhaps.
Re:Apple kills plans to scan for CSAM in iCloud by Terretta on Wednesday December 07, 2022 @10:31PM (#33903808)
These analogies are wrong.

You agree _in your rental contract_ not to dump explosives in the apartment complex trash compactor, and that the apartment complex has the right to process your trash.

The apartment complex stops looking inside everyone's bags that land in the compactor (a bit late to prevent it from getting into the trash), instead sniffing bags as you put them into the trash chute on your floor, and tagging the bag with a red sticker if it the chemical composition matches a particular pre-registered explosive from a list.

At the other end of the chute, a counter checks if you've dumped a dozen bags with red stickers, and if so, a human may open one to see WTF you're up to. If you're indeed dumping explosives in bulk, they let the bomb squad know.

EITHER WAY, that bag would be headed to the compactor, you are affirmatively putting it there, and any explosives are a breach of agreement. In no other case besides you opening the chute to dump a bag is it sniffing anything else anywhere, except when you open the chute to send these bags to the compactor, bags you decide the contents of.

Re:Apple kills plans to scan for CSAM in iCloud by unityByFreedom on Wednesday December 07, 2022 @07:48PM (#33902615)
You'd have a hard time verifying that said on-device scanning would only have been run only on iCloud-uploaded content.

Once the feature exists your local data might become accessible to a government warrant, which would make the iPhone the opposite of a privacy oriented device.

If it's only for iCloud uploaded data they can simply do the scanning there. There's no reason to use customer's CPU/battery against them.

Re:Apple kills plans to scan for CSAM in iCloud by lathiat on Wednesday December 07, 2022 @08:07PM (#33902766)
This necessitates a workflow where both the photos and decryption keys are accessible by the same server and that there is a security workflow to request the users decryption keys without the user involved.

This is specifically what Apple is trying to avoid - they are intentionally pushing an environment where the user must be involved to get the keys, by way of their account password and/or other enforcement mechanisms designed to ensure only the real user can access such keys.

This is discussed in detail in their Apple Platform Security guide: https://help.apple.com/pdf/security/en_US/apple-platform-sec...

All of the facial recognition, object identification, etc is all done on-device for the same reason. By contrast Google can and does do this in the cloud - and there are Google servers that intentionally have access to decrypt your photos.

iCloud backup was previously a vector that bypassed this, however they also today announced they are fixing that: https://news.ycombinator.com/item?id=33897793 "Apple introduces end-to-end encryption for backups"

Re:Apple kills plans to scan for CSAM in iCloud by _jal on Wednesday December 07, 2022 @08:24PM (#33902890)
Sure, from a technical perspective, it is a nice solution to a set of problems.

But there are many serious problems with it. I'll isolate one: an on-device content surveillance mechanism is a slippery slope to a bad, bad place.

As the saying from the 90s goes, "child porn [CSAM] is the root password to the US Constitution." It is bad enough that people are willing to suspend their better judgement to do something about it.

But after you already have the mechanism accepted an in place, it is far easier to add to it. Pick your boogeyman: you're giving all of them a lovely tool to address their desires. Dissent suppression and Winnie-the-Poo detection for Xi, Erdogan gets to sniff out Golem memes, and choose-your-own horror for the coming dictator of the US.

It is far harder to tell a sovereign, "we could easily do that, but will not" than "we don't have a mechanism to do that." And pretending that it won't happen doesn't pass the laugh test - we have seen this show many, many times. But if you want to argue, start by explaining how Apple's jumping to implement the 10-minute-max sharing limit shows how they'd stand up to China about this.

Re:Apple kills plans to scan for CSAM in iCloud by lathiat on Wednesday December 07, 2022 @08:32PM (#33902940)
I agree there is a slippery slope concern, and Apple has themselves made related arguments such as the FBI case and not wanting to create a software update to decrypt the contents of a phone. However that is contrasted with a very real and much more practical concern of malicious parties getting access to your cloud stored photos.

It would help to note Apple also today announced "Advanced Data Protection" in iCloud which closes the hole where iCloud Backups, iCloud Photos and various other bits of data were technically decryptable by Apple. They've closed that (but it's opt-in, to balance the average user losing all their photos against other users desire to be secure even if it means losing their data). Details: https://support.apple.com/en-us/HT202303#advanced

However even without the "Advanced Data Protection" what I said about not having a workflow for any Apple server to "normally" request both the keys and photo data is also still good security.

Re:Apple kills plans to scan for CSAM in iCloud by ec109685 on Thursday December 08, 2022 @01:50AM (#33905009)
To detect Winnie-the-poo, it required a code push of a new database to all clients. If that’s the bar, than a corrupt apple could also push a software update tomorrow that enabled such scanning, whether this scheme was implemented or not.
Re:Apple kills plans to scan for CSAM in iCloud by theshrike79 on Thursday December 08, 2022 @02:49AM (#33905370)
> If it's only for iCloud uploaded data they can simply do the scanning there.

This is what Apple was trying to avoid. Scanning on iCloud also requires that Apple can see your photos.

If the scanning is done on device, Apple could encrypt the photos in the cloud so that they can't decrypt them at all. Neither could the authorities.

> There's no reason to use customer's CPU/battery against them.

The amount of processing ALL phones do for phones is crazy, adding a simple checksum calculation in the workflow does fuck-all to your battery life. Item detection alone is orders of magnitude more CPU/battery heavy.

Re:Apple kills plans to scan for CSAM in iCloud by brookst on Wednesday December 07, 2022 @08:34PM (#33902947)
> Once the feature exists your local data might become accessible to a government warrant, which would make the iPhone the opposite of a privacy oriented device

Why does that dystopia require on device scanning? Why couldn't they just do it with an OS update today? It's not a reasonably slippery slope, given the actual mechanics of how the CSAM system was designed (perceptual image hashes, not access to arbitrary files)

> There's no reason to use customer's CPU/battery against them.

That's the better argument, but still not super strong. On-device scanning means you know and can verify what hashes are being scanned for, who is providing them, and when they change. Cloud scanning is a complete black box. None of us would know if Google was doing ad-hoc scans of particular users' photos at the behest of random LEOs.

Re:Apple kills plans to scan for CSAM in iCloud by unityByFreedom on Wednesday December 07, 2022 @10:37PM (#33903858)
> Why couldn't they just do it with an OS update today?

See my comment here:

https://news.ycombinator.com/item?id=33903825

> None of us would know if Google was doing ad-hoc scans of particular users' photos at the behest of random LEOs.

Not your device, not your software. You should assume anything you upload unencrypted is scanned. This distinction was clearly voiced by the majority during the debacle of Apple's on-device scanning proposal. They basically said, "Scanning in the cloud is [choose one: fine, skeezy], but we draw the line at doing on-device scans. I don't want that software on my device."

Re:Apple kills plans to scan for CSAM in iCloud by tshaddox on Wednesday December 07, 2022 @10:02PM (#33903608)
> You'd have a hard time verifying that said on-device scanning would only have been run only on iCloud-uploaded content.

That would be precisely as difficult to verify as verifying whether your Apple device is currently scanning your content.

Re:Apple kills plans to scan for CSAM in iCloud by unityByFreedom on Wednesday December 07, 2022 @10:33PM (#33903825)
The point of the feature is to use the data in court cases, which are public record. So word would get out there, via journalist, a whistleblower, etc. They had to make the proposal public before implementing it.
Re:Apple kills plans to scan for CSAM in iCloud by mindslight on Wednesday December 07, 2022 @07:33PM (#33902477)
The crazy part is forcing a user to run software that undermines their own interests, on a device they purportedly own that is likely a huge part of their life. Software running on a computer should represent that computer owner's interest, period.

Apple can run software that represents Apple's interests on their own servers.

Re:Apple kills plans to scan for CSAM in iCloud by theshrike79 on Thursday December 08, 2022 @02:54AM (#33905407)
What part of comparing checksums of images on your device to KNOWN child pornography checksums is "undermining your interests"?

This was not some Google-style hotdog or not algorithm that was using "AI" to guess if a photo had a nude person on it or not.

Re:Apple kills plans to scan for CSAM in iCloud by mindslight on Thursday December 08, 2022 @11:10AM (#33909482)
First, the huge possibility of false positives. They're not "checksums", but rather "perceptual hashes", which are nowhere near as solid as a cryptographic hash. When you multiply a small chance of a false positive across hundreds of millions of deployed devices, you end up with many false positives - false positives that result in people being characterized as "child abusers".

Second, the overall aim of the technology also targets legitimate or inadvertent activities, that the law has also unjustly criminalized - say taking family pictures of your kids, 17 year olds sexting, or someone trolling you. Once again unjustly branding people as "child abusers", this time with "evidence" to back up the narrative.

But even in the case of a proper match to the NCMEC list for its bona fide purpose, that still is undermining the phone's owner's interests. That's the philosophical contour, even if you personally wish to brush it aside with the desire to catch people looking at evil images. Our society respects similar longstanding privacy boundaries, even if it does end up helping some "bad people".

It's really not in the NCMEC's interest to respect any of this, as their foundational dynamic is that there is a horrible thing happening in the world, and they must do everything possible to stop it. That kind of advocacy is certainly needed, but we shouldn't just accept their desires as if they're an unbiased neutral party.

Re:Apple kills plans to scan for CSAM in iCloud by theshrike79 on Friday December 09, 2022 @03:39AM (#33919057)
Yes, the internet went on a "generate false perceptual hash positives" binge.

Everyone just forgot that in the Apple system you don't get automatically banned and reported to the authorities if you get a match (or five). The matches get manually checked by an actual human, who will see in 0.2 seconds that it's not actual CP, but a highly distorted picture of a cat or some gibberish. Zero action will be taken.

Also: the scanning was only done if iCloud is enabled. If iCloud is enabled, the EXACT SAME scan could be done in the cloud. Why are people not objecting to this with the same fervour?

Re:Apple kills plans to scan for CSAM in iCloud by saagarjha on Thursday December 08, 2022 @05:44AM (#33906495)
1. What's to guarantee that the checksums are of child pornography?

2. What's going on with the "fuzzy comparison"?

Re:Apple kills plans to scan for CSAM in iCloud by theshrike79 on Thursday December 08, 2022 @06:07AM (#33906635)
The database is generated by the NCMEC[0]. I trust them to have a rigorous process for adding stuff to it.

[0] https://en.wikipedia.org/wiki/National_Center_for_Missing_%2...

Re:Apple kills plans to scan for CSAM in iCloud by saagarjha on Thursday December 08, 2022 @06:27AM (#33906752)
A lot of the concern is that others may not trust the NCMEC, or that they don't trust that other images won't be added in whatever ends up on your device.
Re:Apple kills plans to scan for CSAM in iCloud by infotogivenm on Thursday December 08, 2022 @05:11AM (#33906276)
It took me a while of ruminating but I think I like this answer the best. Fundamentally here some people are upset that something they own would betray them.

Which is weird, because it implies anything you used but didn’t buy can and should betray you, and it’s your own fault. You should have been smarter. But it makes a bit of sense, for some odd reason I can’t quite make tangible. When the samsung TV I bought starts showing me ads, when my pricey cable or Hulu package interrupts for commercials, I feel a tinge of this.

So now I think I somewhat get the feeling. Thank you.

Re:Apple kills plans to scan for CSAM in iCloud by mindslight on Thursday December 08, 2022 @10:51AM (#33909181)
> because it implies anything you used but didn’t buy can and should betray you, and it’s your own fault

The contrapositive isn't automatically true. You can be upset when a cloud host or ISP violates your expectations, even though you don't own them. You can expect Apple to basically work in your interests even on their own servers, since you're paying them.

The ownership thing is more about the structure of society. If computing and communications technology is going to move us forward as a distributed democratic society, then those capabilities must remain distributed throughout the population rather than being controlled by a handful of centralized gatekeepers.

And yes, we've strayed quite far from this ideal. Most people have little control over what their mobile phone does. Still, there is a distinction between a device not having your desired functionality because the manufacturer didn't create it (or even disallowed others from creating it), and the manufacturer deliberately creating functionality that harms users.

Re:Apple kills plans to scan for CSAM in iCloud by brookst on Wednesday December 07, 2022 @08:36PM (#33902965)
Meh. Running it on device means that it is narrowly scoped and transparent. There is no way to secretly add a new hash or to vary what you're searching for depending on a user's location or identity.

Running it on their servers means nobody knows how expansive the scans are, and there is nothing preventing ad-hoc scans for arbitrary content for individual users.

Re:Apple kills plans to scan for CSAM in iCloud by ec109685 on Thursday December 08, 2022 @01:54AM (#33905027)
The software uploaded an encrypted fingerprint as metadata as part of every photo.

The actual scanning and apple self-interest all happened server side.

Re:Apple kills plans to scan for CSAM in iCloud by mindslight on Thursday December 08, 2022 @11:29AM (#33909769)
Truthfully generating that fingerprint is not in the user's interest. User representing software would skip doing that. If the storage service required it, it would generate plausible dummy values.
Re:Apple kills plans to scan for CSAM in iCloud by sitzkrieg on Wednesday December 07, 2022 @09:12PM (#33903215)
users havent owned devices in quite a while in this space
Re:Apple kills plans to scan for CSAM in iCloud by TechBro8615 on Wednesday December 07, 2022 @07:48PM (#33902614)
Pretty much every "file" you create on your iphone is uploaded to iCloud by default. For example if you take a screenshot, it saves to your photos library which defaults to uploading new photos to iCloud immediately.
Re:Apple kills plans to scan for CSAM in iCloud by endisneigh on Wednesday December 07, 2022 @08:10PM (#33902790)
The same files would’ve been scanned anyway. What’s the problem?
Re:Apple kills plans to scan for CSAM in iCloud by archagon on Thursday December 08, 2022 @04:08PM (#33913431)
It's my device, working against me; and it's also not going to catch anyone remotely intelligent, since everyone and their mother will know about this feature.