News for hackers. Stuff that matters.
Re:Apple kills plans to scan for CSAM in iCloud by GeekyBear on Wednesday December 07, 2022 @08:20PM (#33902860)
Scanning in the cloud can be just as problematic.

Google is not just scanning for known examples of kiddie porn, they are scanning for images of naked children, and reporting parents to the police.

for instance, they tried to get a parent arrested for using their Android phone to get medical assistance for their child during lockdown.

>With help from the photos, the doctor diagnosed the issue and prescribed antibiotics, which quickly cleared it up. But the episode left Mark with a much larger problem, one that would cost him more than a decade of contacts, emails and photos, and make him the target of a police investigation.

“I knew that these companies were watching and that privacy is not what we would hope it to be,” Mark said. “But I haven’t done anything wrong.”

The police agreed. Google did not.

https://www.nytimes.com/2022/08/21/technology/google-surveil...

Re:Apple kills plans to scan for CSAM in iCloud by autoexec on Wednesday December 07, 2022 @08:44PM (#33903022)
I wouldn't be surprised if Microsoft wasn't doing the same thing with photos viewed in their default Photo app (The names of the images viewed in that app are sent as telemetry already along with the times they were viewed and how long each image was left open).

Windows defender may even be checking the files it scans for whatever content MS considers objectionable. We likely wouldn't won't know unless it becomes a news story or a whistleblower tells us.

Re:Apple kills plans to scan for CSAM in iCloud by GeekyBear on Wednesday December 07, 2022 @09:09PM (#33903198)
> I wouldn't be surprised if Microsoft wasn't doing the same thing with photos viewed in their default Photo app

Google has apparently shared their algorithm with others, but only Facebook was mentioned by name.

>in 2018, Google developed an artificially intelligent tool that could recognize never-before-seen exploitative images of children. That meant finding not just known images of abused children but images of unknown victims who could potentially be rescued by the authorities. Google made its technology available to other companies, including Facebook.

https://www.nytimes.com/2022/08/21/technology/google-surveil...

Re:Apple kills plans to scan for CSAM in iCloud by BuyMyBitcoins on Thursday December 08, 2022 @12:31AM (#33904553)
I had no idea the photos app was doing this. This seems beyond excessive and I’m glad I happen to use the old photo viewer. The new one always takes a second to load the photo, and I suppose that’s because it’s doing all this other stuff in the background.
Re:Apple kills plans to scan for CSAM in iCloud by ls15 on Thursday December 08, 2022 @04:57AM (#33906200)
I often hear the argument that the average user does not care about privacy, but I don't think that this is true.

Instead I think the average user is just not aware of the insane amount of data that is exfiltrated and what can be done with that data, because the software does not make it clear what it actually is doing.

Re:Apple kills plans to scan for CSAM in iCloud by BuyMyBitcoins on Thursday December 08, 2022 @09:10AM (#33908016)
Honestly, why on Earth does Microsoft want to know the filenames of the photos being opened? And, how often each one is viewed? What possible enhancement to the app can be justified because of this information?
Re:Apple kills plans to scan for CSAM in iCloud by autoexec on Thursday December 08, 2022 @09:42AM (#33908332)
Cynically, I'm sure it could be "justified" in all kinds of ways like "knowing more about the photos viewed most often with our Photos app helps us to determine what new features would best enhance the user experience. Only metadata is being collected, so no employee is ever able to view your pictures. We value your privacy blah blah blah"

Even more cynically, it just gives them more data about you and your interests (with viewing frequency/duration being a measure of interest) which lets them push more narrowly targeted ads at you, which gives them more money, which lets them invest more in improving Windows as a whole or whatever else they tell themselves so they can face their own reflections.

Re:Apple kills plans to scan for CSAM in iCloud by Eisenstein on Thursday December 08, 2022 @01:54AM (#33905029)
I checked out this claim and and was able to find this:

* https://www.reddit.com/r/Windows10/comments/8zk1yy/a_simple_...

It stores that information in a local index, and also stores telemetry data which counts interactions with the app, but there is no evidence it is uploading indexes to MS.

Do you have any evidence of this?

Re:Apple kills plans to scan for CSAM in iCloud by autoexec on Thursday December 08, 2022 @06:01AM (#33906597)
I believe that info was in one of these links:

https://learn.microsoft.com/en-us/windows/configure/windows-...

https://learn.microsoft.com/en-us/windows/configure/basic-le...

They both 404 now. I had saved a copy of both at the time, but those files must be on another drive. I'll keep looking though!

There were several articles about it at the time https://www.extremetech.com/computing/247311-microsoft-final...

https://www.ghacks.net/2017/04/06/windows-10-full-and-basic-...

These documents were massive and filled with lots of information buried in places that seemed odd and scattered across different sections so that you couldn't get all the information about what an app collected in one section/table of the document.

Another thing I remember from those pages was that in some circumstances short clips (less than 1 minute) of the videos you played using window's default player would be sent to MS as well, but they said that would only happen in the event of crashes.

Re:Apple kills plans to scan for CSAM in iCloud by soulofmischief on Wednesday December 07, 2022 @09:14PM (#33903237)
Even if it isn't currently checking against CSAM databases, it does by default share file metadata with Microsoft.
Re:Apple kills plans to scan for CSAM in iCloud by 9991 on Thursday December 08, 2022 @05:03AM (#33906238)
If you're still using Microsoft or Google services at this point, you deserve what you get.
Re:Apple kills plans to scan for CSAM in iCloud by deely3 on Friday December 09, 2022 @02:57AM (#33918819)
Victim blaming is very easy, yes?
Re:Apple kills plans to scan for CSAM in iCloud by 9991 on Saturday December 10, 2022 @08:21AM (#33932386)
Yes, pretty easy. Keyboard-warrioring in general is.

But that's not the point. Victim blaming doesn't do any good for today's victim, but maybe it will help show the next person that their fate is under their control.

Re:Apple kills plans to scan for CSAM in iCloud by knaik94 on Wednesday December 07, 2022 @08:47PM (#33903053)
If the photos are hosted on Google's servers, they have every right to scan whatever they want. If you are privacy conscious, you can upload files you encrypt yourself at the cost of not being able to share easily. I recognize edge cases like the one in the article you linked, but I don't see an alternative. Not scanning for CSAM on your own servers isn't a realistic expectation.
Re:Apple kills plans to scan for CSAM in iCloud by GeekyBear on Wednesday December 07, 2022 @09:03PM (#33903155)
> If the photos are hosted on Google's servers, they have every right to scan whatever they want.

Google is trying to get people arrested because Android backed up their personal photo libraries to Google Photos.

> The day after Mark’s troubles started, the same scenario was playing out in Texas. A toddler in Houston had an infection in his “intimal parts,” wrote his father in an online post that I stumbled upon while reporting out Mark’s story. At the pediatrician’s request, Cassio, who also asked to be identified only by his first name, used an Android to take photos, which were backed up automatically to Google Photos. He then sent them to his wife via Google’s chat service.

https://www.nytimes.com/2022/08/21/technology/google-surveil...

> Not scanning for CSAM on your own servers isn't a realistic expectation.

Google isn't just scanning for known examples of kiddie porn, they have developed an algorithm that scans for any image of naked children and, as usual, there doesn't seem to be a human in the loop when their algorithm makes a bad call.

Re:Apple kills plans to scan for CSAM in iCloud by knaik94 on Wednesday December 07, 2022 @09:21PM (#33903285)
It's Google's servers. I imagine every content hosting company has a heuristics based approach to detecting CSAM. Google has a habit of tuning moderation algorithms for false positives rather than missed positives.

Not having a human in the loop is terrible and a problem frequently when it comes to getting support from Google companies. That's a separate issue from Google or Dropbox being able to scan unencrypted files for CSAM. Google's policy of automating as much as it can has tons of downsides. But it's understandable when you look at the scale Google functions.

It's important to separate the policy of scanning from Google's terrible appeal process and the algorithm false positives.

I would feel differently if the story was that Android scanned an outgoing SMS or a photo saved locally. I am not sure where the balance point is to identify and report CSAM while also respecting user's rights to privacy.

Re:Apple kills plans to scan for CSAM in iCloud by GeekyBear on Thursday December 08, 2022 @06:45AM (#33906868)
> It's Google's servers. I imagine every content hosting company has a heuristics based approach to detecting CSAM.

The slippery slope claim made in the past was that they were only looking for files that match known examples of kiddie porn.

For instance, this tech from Microsoft:

>PhotoDNA creates a unique digital signature (known as a “hash”) of an image which is then compared against signatures (hashes) of other photos to find copies of the same image. When matched with a database containing hashes of previously identified illegal images, PhotoDNA is an incredible tool to help detect, disrupt and report the distribution of child exploitation material.

https://www.microsoft.com/en-us/photodna

Reporting parents to police because they took pictures of their child's first bath and those photos were automatically backed up to Google servers is much, much worse.

Re:Apple kills plans to scan for CSAM in iCloud by Eisenstein on Thursday December 08, 2022 @09:29AM (#33908208)
Please stop using 'child's first bath' as an indicator of overreaction by the algorithm. It is not helpful to argue something which will make you look hyperbolic or histrionic once your argument is looked into.

1. The picture(s) in question mentioned above which got an account shut down and police notified was not a 'child in a bath', it was close ups of a child's genitals

2. You are probably not a parent since a child's 'first bath' is not a thing. They are bathed as infants starting immediately. If you are referring to a child's first independent bath, then no one should be taking pictures since private bathing is not something that should be intruded upon for picture taking

All that said, I agree with you.

Re:Apple kills plans to scan for CSAM in iCloud by ryanobjc on Thursday December 08, 2022 @10:52AM (#33909186)
> 2. You are probably not a parent since a child's 'first bath' is not a thing. They are bathed as infants starting immediately. If you are referring to a child's first independent bath, then no one should be taking pictures since private bathing is not something that should be intruded upon for picture taking

Are you a parent? A prudish parent perhaps?

I have pictures of me as a kid in a bath (genitals obscured underwater). It's a common parental thing among parents who AREN'T DRIED UP PRUDES AND FRIGHTED OF NUDITY.

The bottom line is - another case of our ability to use our technology as we desire being interrupted and interfered with. I hate to even remotely invoke RMS, but... He wasn't wrong!

Re:Apple kills plans to scan for CSAM in iCloud by Eisenstein on Thursday December 08, 2022 @01:18PM (#33911158)
Please re-read what I wrote. Was it really unclear? I said that if a child is bathing independently that it is not appropriate to barge in and take pictures of them. They would be at least 5 or 6 years old at that point. Is that prudish? Why is everyone so emotional about this? All I said was that it was a terrible and inaccurate argument to talk about pictures of a child in a bath because that is not what flagged the algorithm and there is no such thing as a child's first bath.

Please try to parse a response before yelling at someone for something they didn't state.

Re:Apple kills plans to scan for CSAM in iCloud by GeekyBear on Thursday December 08, 2022 @10:14AM (#33908670)
> 1. The picture(s) in question mentioned above which got an account shut down and police notified was not a 'child in a bath', it was close ups of a child's genitals

No, the example of Google calling the police on a parent given in the NYT article was because the parent sought medical assistance for their child during lockdown and the doctor requested pictures of the issue the child was having.

This is no more an acceptable reason for Google to report parents to the police for child abuse than taking pictures of your child's first bath, and taking pictures of your child's milestones has long been something that people have done.

Re:Apple kills plans to scan for CSAM in iCloud by Eisenstein on Thursday December 08, 2022 @10:29AM (#33908866)
>> 1. The picture(s) in question mentioned above which got an account shut down and police notified was not a 'child in a bath', it was close ups of a child's genitals

> No, the example of Google calling the police on a parent given in the NYT article was because the parent sought medical assistance for their child during lockdown and the doctor requested pictures of the issue the child was having.

These things are not mutually exclusive.

Stop being defensive. It doesn't help you. I am trying to help you by telling you that you are making a shitty argument.

Re:Apple kills plans to scan for CSAM in iCloud by GeekyBear on Thursday December 08, 2022 @11:06AM (#33909422)
> I am trying to help you by telling you that you are making a shitty argument.

OK. Allow me to tell you that, in my opinion, you are the person making the shitty argument

Your argument that "no one should be taking pictures" of their child's first bath is just cringe-worthy.

Google is to blame for the situation when they call the police over a single false positive from their faulty algorithm, not parents who take pictures of their own children.

Re:Apple kills plans to scan for CSAM in iCloud by Eisenstein on Thursday December 08, 2022 @01:16PM (#33911130)
Are you even reading what I am writing? I never said that no one should take pictures of their children's first bath -- I said that there is no such thing as a child's first bath, just like there is no such thing as a child's first meal, or a child's first nap, or a child's first bowel movement. They are given baths as infants.

You are obviously either unable or unwilling to parse what I am trying to tell you, and fixated on this notion that you have to be 'right' no matter what even though I am not telling you that you are wrong!

I am at a loss as to what to reply to you with besides I urge you to read things before continuing to respond because otherwise you end up talking around people instead of with them.

Good day.

Re:Apple kills plans to scan for CSAM in iCloud by ryanobjc on Thursday December 08, 2022 @10:49AM (#33909148)
How do you take a photo on android to only store it locally?

The whole photos experience on Android has been simplified to break that distinction and there is no way to even really know or realize what is happening - BY DESIGN.

The real answer here is, again, Android is not privacy first by default.

BTW parents taking medical pictures of their children != CSAM which is an acronym for "Child Sexual Abuse Material"... Taking care of your child's medical problem isn't "sexual abuse" despite the conflation of the two people are glossing over in this thread.

Re:Apple kills plans to scan for CSAM in iCloud by knaik94 on Thursday December 08, 2022 @03:30PM (#33912918)
I installed an offline gallery and disabled Google Photos. For years and years Samsung android phones didn't have the option of cloud backups.

There's a very clear distinction between local and cloud, every single photo taken using any camera app is local only. Galleries, like Google Photos, and backup services like Dropbox have an explicit setting to enable cloud backup. Google Photo backup is very distinctly different from Google Drive and phone backups in general.

I use FolderSync to sync it with a self hosted NextCloud instance. If I am travelling, I prefer using Syncthing, a wifi file server running off the phone, or a physical wire to transfer photos. I know exactly what pictures are sent where and backed up with what method. I am not sure why you believe Android treats photo and file management like a black box, I don't believe Apple does either. Apple iOS devices are much more difficult to manually backup as they only allow background photo sync to iCloud, if the screen is turned off syncing to a third party will most likely fail. But there is a straightforward toggle to disable Photos backup to iCloud.

Android file management is superior to any other mobile OS. Android is file privacy first by design, as long as you are comfortable managing backup and syncing with self-hosted and self-managed solutions. I have gone the additional step of rooting my phone as well to get around any Android limitations on what folders certain apps have access to.

No one is arguing against the last point you made. No one was conflating the two.

Re:Apple kills plans to scan for CSAM in iCloud by GeekyBear on Friday December 09, 2022 @02:19AM (#33918578)
> There's a very clear distinction between local and cloud, every single photo taken using any camera app is local only.

>Android is file privacy first by design

Yet we have the NYTimes article stating that the only ways Google could have access to the photos they tried to have a parent arrested for would be to scan content on the device, scan automatic backups, or spy on the text messages sent from the device.

The examples in the story were parents sharing images directly with their doctor, not the with the general public.

Re:Apple kills plans to scan for CSAM in iCloud by nanidin on Wednesday December 07, 2022 @11:31PM (#33904178)
I wonder how algorithms like Stable Diffusion will impact CSAM detection policies.
Re:Apple kills plans to scan for CSAM in iCloud by Sporktacular on Thursday December 08, 2022 @09:47AM (#33908371)
If a child isn't exploited to make it, it's not technically CSAM.

But you could maybe argue that if the ML model is based on CSAM then it's a product of it and would need to be banned.

Re:Apple kills plans to scan for CSAM in iCloud by shitlord on Thursday December 08, 2022 @04:25AM (#33906013)
Google scans your data for anything it finds objectionable. Here's another example of Google flagging and deleting a video from a user's Drive storage: https://twitter.com/nicholasdeorio/status/158386625133969818...
Re:Apple kills plans to scan for CSAM in iCloud by GeekyBear on Thursday December 08, 2022 @07:04AM (#33906969)
There are many examples of Google's automated systems making egregious mistakes while scanning user data with no human in the loop to review the decision.

>Ed Francis studies the evolution of military technology over at his YouTube channel, Armoured Archives. But this week, Francis says five years’ worth of research stored on Google Drive has become inaccessible thanks to Google’s automated error.

Francis says the file in question was simply a collection of data on various tanks for a coming video on how military vehicles have evolved across historical conflicts. But Google’s automated systems deemed the file a terrorist threat, resulting in a complete lockdown of his YouTube, GMail, and Google Drive accounts.

https://www.vice.com/en/article/qj8yj7/google-locks-historia...

Having a shitty algorithm kill your whole Google account with no way to reach a human to fix the problem is one thing.

Having a shitty algorithm report you to the police for taking pictures of your child's first bath is a bridge too far.

Re:Apple kills plans to scan for CSAM in iCloud by knaik94 on Thursday December 08, 2022 @03:06PM (#33912590)
I remember that story, I came across a post asking people to upvote the support ticket on the r/datahoarder subreddit by a fan on the 17th. I came across it and cross posted a link on hacker news on the 22nd in the evening and by the following morning the account was restored and taken care of. I can't say for sure how much posting it here helped, but based on the timeline I think it made a difference. I believe it was also one of the highest upvoted stories of that day.

What stood out to me what one of the first couple comments in the support thread was the contact information for the Museum Director of the Swedish Tank Museum.

My unconfirmed theory is that Google's OCR pdf service flagged specific text and pages in the pdf of tank plans and repair manuals as they are considered classified and shouldn't be in the public unredacted. It's historically significant and absolutely worth preserving, but I can see why it might get flagged.

This was also interesting in that it got the entire account taken down. Usually Google flags a file and disables sharing completely. Google disabled sharing backups of a recent Kanye interview that was filled with hate speech called "Kanye champs removed video.mp4". I have not watched it but it was my first time seeing something removed because it "may violate Google Drive's Hate Speech Policy. Some features related to this file were restricted."

I am not sure if it was the post to HN or various videos other made that helped, but it made me realize that the one of the best ways to get in touch with a human at Google is posting here on HN and that Google will continue to become more aggressive with scanning files.

Section 230 was modified by FOSTA-SESTA when related to hosting sexual solicitation, and EARN IT is back under review by congress which would effectively make E2EE illegal and make hosts liable for illegal content uploaded by its users.

Would scanning be okay if there is a government entity with a human you can appeal to that would override any flags made by automated systems? No corporation wants more government oversight, and the only way to avoid it is to do good faith self policing. EARN IT would receive a lot more support if the public sentiment changes to believe Google and Apple and other hosts intentionally choose to ignore problematic materials. Reddit is criticized often because it allowed problematic subreddits to grow. I am not sure where things will end up.

Re:Apple kills plans to scan for CSAM in iCloud by doomslice on Thursday December 08, 2022 @07:33AM (#33907177)
I'm not defending this at all, but one of the reasons why there are no (or few) humans that can be contacted is that they* said that it was tried before and it caused a lot more issues with mistakes/takeovers due to social engineering.

* Can't remember who said it but it was at a town hall this year

Re:Apple kills plans to scan for CSAM in iCloud by GeekyBear on Thursday December 08, 2022 @08:35AM (#33907702)
> one of the reasons why there are no (or few) humans that can be contacted is that they* said that it was tried before

This just sounds like yet another excuse for holding payroll down as much as possible.

If I am a customer of Amazon, Apple, Netflix, Walmart or any number of the other companies with a similar market cap, I can get access to real live human beings who provide customer support.

Re:Apple kills plans to scan for CSAM in iCloud by josefx on Thursday December 08, 2022 @12:07AM (#33904416)
> If the photos are hosted on Google's servers, they have every right to scan whatever they want.

I don't get why that should be the case. When you rent an apartment the owner is generally no longer able to just walk in and do whatever he likes, including installing cameras in the shower. So I would expect that when Google tries to sign you up for its online storage (repeatedly) you should get the same protections, especially with smart phones being such an important part of modern life.

Re:Apple kills plans to scan for CSAM in iCloud by knaik94 on Thursday December 08, 2022 @03:00AM (#33905460)
It's more about shareable media. Relative to your analogy, I feel storing files in encrypted containers is renting an apartment and feeling confident about the landlord not walking in. On the other hand, if you hang up an obscene image in your window of your apartment, I imagine it's understandable that a landlord would use a key and take it down. If hanging up obscene images becomes a pattern, I think the landlord would kick you out.

From one perspective, I understand that cloud storage should include certain rights to privacy. If I buy some compute time on AWS, I manage and control all flow of data. It would be a complete violation if AWS policed the kind of content I could store. I think self-managed vs company managed is what dictates my expectations. Google photos and drive, especially since most people don't pay for it, fall into the company managed category in my opinion.

In the last few years Google Drive became more strict about sharing copyright work and started flagging files, which limits the ability the "share to anyone with a link". For people who use it as a backup, the advice I started hearing was to encrypt the data before uploading. The opsec while considering nation-state level threats to privacy already recommended only using encrypted containers since the beginnings of cloud storage.

The Communications Decency Act is an interesting law that dictates the liability of online content and service providers. There is an ongoing battle to increase ISP and host liability as a way for the entertainment industry to combat piracy.

FOSTA-SESTA and the ruling on Backpage [1] made things more difficult for hosts like Google and changed the liability surrounding sexual solicitation. If a host like Google knows what's being hosted and does nothing, they get in trouble. They don't have the protections they did in the past.

https://en.wikipedia.org/wiki/Section_230

Re:Apple kills plans to scan for CSAM in iCloud by GeekyBear on Thursday December 08, 2022 @06:54AM (#33906919)
> It's more about shareable media... if you hang up an obscene image in your window of your apartment

Again, the parents Google is reporting to the police are not trying to share photos with anyone except their own doctor.

This isn't scanning information because you made it publicly available. This is scanning information on your device because it was automatically backed up by Android.

Re:Apple kills plans to scan for CSAM in iCloud by tpush on Thursday December 08, 2022 @05:36AM (#33906426)
> [...] I imagine it's understandable that a landlord would use a key and take it down.

Just to give context to perspectives people might have on this analogy, the above would be illegal in e.g. Germany.

Re:Apple kills plans to scan for CSAM in iCloud by jliptzin on Wednesday December 07, 2022 @09:04PM (#33903166)
Maybe monitor companies should put chips inside that scan the current screen every second to check for CSAM matches or whatever else they don’t want the customer using their monitors for and diligently report violations to the police?
Re:Apple kills plans to scan for CSAM in iCloud by nanidin on Wednesday December 07, 2022 @11:26PM (#33904147)
You’re joking but smart TV screens already do this to identify and report content back to the mothership.
Re:Apple kills plans to scan for CSAM in iCloud by magicjosh on Wednesday December 07, 2022 @10:10PM (#33903673)
Yes! The Creating Sustainable American Mployment (CSAM) Act will create thousands of new jobs too.
Re:Apple kills plans to scan for CSAM in iCloud by abraae on Wednesday December 07, 2022 @09:17PM (#33903252)
> Not scanning for CSAM on your own servers isn't a realistic expectation

It is if the false positives are overwhelming, given their impact on innocent people's lives.

Re:Apple kills plans to scan for CSAM in iCloud by magicjosh on Wednesday December 07, 2022 @10:15PM (#33903691)
Not scanning for CSAM on your own servers isn't a realistic expectation.

What makes it not a realistic expectation? According to other references, the USA cannot compel companies to run scans on their own customers.