It seems like a problem that requires legal change. Right now Google is fully entitled to nuke the persons email and phone number, completely locking them out of their life. And they can do this for any reason without any recourse.

Once you reach the scale and importance of these services, it shouldn’t be legal to lock people out like this.

It does require legal change but not the one you're implying. I love how the conversation moved from the initial outrage of "they're scanning my private images!" to "ok, they are scanning my images but I need to have a legal recourse to get my accounts back in case I'm banned improperly".

There is no good excuse for this kind of spying apparatus, period. If anything, having a mandated formal process for reinstating banned accounts will make the scanning more acceptable, which it shouldn't be. "You see, yes, we're regularly checking out your dick pics but it's all good, we have a process in place and if we tag an image as CP, when it's not, you'll eventually be exonerated. No harm done"

I would agree with you, except for this which goes too far IMHO:

> If anything, having a mandated formal process for reinstating banned accounts will make the scanning more acceptable, which it shouldn't be.

We should fight first for a proper legal recourse, and once this happens, then we can also fight for no scanning.

Why? Why should be fight for legal recourse first before we tackle the real problem? (At least in this context, legal recourse for unwarranted service termination can also be useful for things unrelated to scanning of private data, but that's not the point) This sounds like the "step back" in the "two steps forward, one step backwards" ideom.

I'd argue because it would likely be an easier battle and would still win rights and, perhaps, set precedent for the limitations of entities like Google here and elsewhere. The alternative of blocking scanning all together is extremely easy to railroad.

>We should fight first for a proper legal recourse

It does not matter, because we don't have any fight left in us at all..

Speak for yourself.

I'm more than prepared to just stop working for a few days or even a week or two.

I already actively boycott Chinese products where possible (I buy non Chinese if at all possible, even at significant higher prices), support the Georgian Legion with my own cash and have probably spent hundreds of hours on unpaid online activism this half year on top of what I do for my church (although in all fairness it still does more for me).

If enough people band together and just plain stop working, sooner or later politicians will have to listen.

I think companies that have sufficiently significant market capture should be laden with a "universal service obligation" as an antitrust-type measure. They shouldn't be able to refuse service unless they can prove the customer is doing something illegal. If they do refuse service, the customer should have grounds to sue them to restore service.

This might go against some people's idea of a free market. I think it should only apply where there isn't a free market - when a customer doesn't have many other options, such as in the case of Google or Apple, or Visa/Mastercard, a major supermarket, or the choice of cellphone provider or Internet provider in markets where there are only a few options. It wouldn't apply to small independent traders by definition.

Capture enough of the market where there isn't enough competition and you get obligations to treat them fairly.

Companies like Google rely heavily on AI and automation and keep humans out of the loop to the extent possible. Once the models detect these sort of photos, they won't be stopping. If they don't add a permanent exception then the AI would flag it on the next pass and repeat the process. Google certainly won't be retraining the models to learn to exclude "but not photos for doctors". Having humans going through and flagging accounts (or photos) for exceptions will ruin the economies of scale.

This can't keep going on like this. To somebody like Google, an account might just be a profit source, but to the affected people it can be their lives or livelihood.

The issue of right of redress really needs to be forced, legislatively.

I wonder whether the man could sue Google for defamation because they've asserted that he's done something and taken an action as a result of that. As a result they're defaming him. I'm not a lawyer and definitely not sure of US law. It'd be interesting to hear a real lawyer consider this.

> a problem that requires legal change

Playing devil’s advocate, possession of CP is criminal with absolute liability [1]. Deleting someone’s account, far from it. (Civil. Maybe.)

Weighing someone who might have CP, even to an absurdly low probability, against them being suing and like two people deleting their Gmail accounts, I can see how someone chose the former. If they got the call wrong, Google would not only be liable for damages, the individuals involved could be criminally charged.

I’m assuming everyone in this chain would appreciate laws clarifying an email bill of rights. On the other hand, Google made their bed with its surveillance-first culture.

[1] https://en.m.wikipedia.org/wiki/Absolute_liability

You are playing devil's advocate against the current legal situation. I don't agree, but fine. But you're responding to the suggestion to make a legal change, this legal change presumably would put a liability on these kinds of false positives too. One of the two could still have stiffer sanctions, and cause Google to prefer the other, but the idea is at least that this choice wouldn't be that obvious anymore.

I agree with you. I’m arguing one could likely get Google to back a reasonable proposal.

> they got the call wrong, Google would not only be liable for damages, the individuals involved could be criminally charged.

So how come this has never happened?

Google and Facebook have existed for decades, and so has CP, they didn't have automatic detection untill decently and yet none of their employees have ever been charged.

I've got a question for the lawyers here. These photos were for communication with a medical professional. Can't the argument be made that this was a HIPAA violation? Isn't this grounds for an entire whale of a class action lawsuit ?

IANAL, but my understanding is HIPAA applies to covered entities, i.e. medical professionals.

Google can use the defense that somewhere in the TOS it probably says you agree to share everything with them when it goes to cloud storage.

HIPAA doesn't prohibit people from sharing info about themselves (even accidentally). Though it does probibits your doctor from putting photos like this near Google anything.

HIPAA specifies how medical professionals should handle patient information. None of the parties in this dispute are medical professionals, so HIPAA does not apply.

No. Google Photos under a personal Google account is not HIPAA compliant and doesn't claim to be.

I guarantee that the doctor has a HIPAA agreement that you sign when you agree to communicate over email that says that you grant him the right to share the files with Business Associates, and that their email providers are considered Business Associates for the purpose of transmitting ePHI.

It sounds more like they say "We're wrong and your account remains banned."

How could they conceivably even look at the data to unban the account though? That shit's radioactive, as soon as the AI flags it I don't know how a (non-LEO) human could possibly view it without running afoul of the law. There's no reasonable way to reverse this action.

How can it be a match if these CSAM scanners are only comparing against a database of known material? Have we been lied to?

I raised this point a while back in another discussion and was assured this kind of thing couldn't happen:

https://news.ycombinator.com/item?id=32178108#32178137

The biggest worry about these kind of situations is not that mistakes are made, but that it's almost impossible to make contact with a real human being to put things right [even, as in this case, when press and police are involved]. It's only a matter of time before someone loses their livelihood or even their life over this kind of cock-up.

It's also easy to lose your livelihood or even life if you fail to eject something from your company's possession after you have reason to believe it contains certain materials of abuse. Federal and state prosecutor opinions differ, so clearance from local or state LEO (as done here) basically means dick. If you're wrong, then you get thrown in prison where you hear stories of "chomos" getting raped and shanked. If google is right, maybe they get a little bit of advertisement money. Imagine if this father did something later and the prosecutor uses this happening as evidence that this content was 'knowingly' harbored by the company. Seems like a bad calculus to me, just cut the guy loose whether he is innocent or not. You don't have the same rights to private service as you do rights in public.

The article says it's a combination of hash matching, AI and human review.

thats equivalent to not saying anything

Just because some systems only do such comparisons (e.g. what Apple had in their iPhone-scanning proposal) doesn't mean all of them work that way.

> There's no reasonable way to reverse this action.

How about "oh, everyone says we are wrong click to unban"?

> How about "oh, everyone says we are wrong

Not even "everyone".

How about "law enforcement that we reported it to says we are wrong"?

'local government/LEO gave the all clear' is not a tight defense against federal prosecution and conviction. For instance, Jeremy Kettler was convicted despite full local and state blessing for possession of a silencer in violation of federal law (he, the state, and local government all thought he was following all laws including federal ones, but it turned out federal prosecutor had a different opinion on what 'interstate commerce' is). I personally would never risk getting registered as an offender on the hopes some federal prosecutor has a different view than local LEO, it's just not worth unbanning the guy unless both the state and federal prosecutor have signed statements granting immunity from prosecution.

Non LEO are routinely employed by companies like Google, Facebook etc to review content. It's the most thankless job in the world.

I don't think it's legal for those people to view it though if they have reason to believe it contains certain images of child abuse. Once the AI flags it that shit goes radioactive and is basically unviewable until LEO looks at it. Even if local LEO looks at it and clears it, that doesn't clear you from being federally prosecuted if you review the content later.

Here is how it worked when I ran a file sharing service:

- We would allow people to upload files

- these files would go into a 'holding tank'

- the files would then get greenlit if some rudimentary algorithm determined they were landscapes or other innocent content

- the remainder would be flagged for review

- bulk images would be displayed in a grid of 8x8 thumbnails

- the vast majority of those would not be reason for further action and would be greenlit, the remainder would go into yet another holding tank

- now the pictures would be viewed full size, any kind of image that would be against the TOS would be rejected and a hash would be kept to avoid seeing it again

- any kind of image that was deemed illegal or borderline would be passed on to LEO for another decision, possibly resulting in criminal charges against the uploader.

The complications:

- uploaders would try to mask their identity

- uploaders would be all over the world

- even after reviewing LEO would sometimes indicate that an image was inconclusive (in those cases we erred on the side of caution)

--

In short: companies can and do work together with law enforcement to stop service abuse and illegal activities. Whether it is an AI that flags it or a human is from a legal perspective utterly immaterial, and if LEO clears it that definitely means that you are allowed to either view it or pass it on.

If AI flag and human is "immaterial" difference as you noted, wouldn't AI flagging it as abuse material make it impossible to view it by a (non-LEO) human afterwards, as now that would show actual intent to view that material (it is illegal to view it with intent to view it).

Surely your reviewers never deliberately looked at the material a second time after marking it as abuse material for law enforcement, as that would be legally risky I would think.

The issue here is even if local LEO clear it, the federal prosecutor's opinion can differ and he can use the fact you knew AI flagged it and that you released it anyway to damn you. There are cases out there where local/state government cleared someone of wrong-doing but they still got hemmed up federally.

LEO = Law Enforcement Officer?

cmiiw

Thanks. Works better than Low Earth Orbit.

In a way, I'm actually glad this is the case.

The "blue check fix" minimizes negative publicity while doing nothing for 99% of people. If a bad policy is consistently applied, at least it can be understood and criticised.

If they’re not right, but they deleted his data, they may have a bigger problem

Why? Isn't it always part of a TOS that you have no rights and that the company can do whatever they like anytime they like?

Contracts or terms don't have to be legal

Surely a GDPR violation at the very least

He could probably still get his data as a data dump if he was European, just not his accounts reinstated. As Mark is apparently an American citizen though, the GDPR does not apply.

Does anyone know what happens if one is a European citizen (or dual-citizen with the US) but physically resident in the US?

Can one invoke EU rights, or is it tied to being physically resident like a DVD region?

What if the EU citizen flies to Europe? Can they then invoke their right from the physical territory of Europe?

I feel this is somewhat uncharted water.

Google has a policy that they delete permanently the contents of any account that has been suspended for 30 days or more. They do this to comply with European GDPR regulations (which require they not hold any data they no longer have the users permission to store, nor have ongoing legitimate interest in), and they have written it into their own privacy policies.

Therefore, this account and all the data contained is already gone. The most they could do is allow him to create a new account with the same name, but to my knowledge that isn't supported technically in their systems (too many bugs to be found by re-using unique identifiers).

That's why they stand by their decision - they don't have the capability to implement any other option.

> They do this to comply with European GDPR regulations

In this case, they actually satisfy the two requirements:

1. "ongoing legitimate interest". They are still in a dispute with the customer. Deactivating an account does not require deleting data, so long as the case it not closed. There is no "30 day" limit in this case, and it seems like an arbitrary time period imposed by someone who hasn't thought of the consequences.

2. "users permission". In this case they do have the user permission to keep the content, because the user wants to keep access to the account.

> they don't have the capability to implement any other option.

As mentioned above, you can deactivate an account without also deleting it. It may of course be easier for Google to just delete it instead of introducing a new state, but I would argue that a company such as Google does have the capability to implement this option. I would go so far as to say this is a failure of Google to not have such a process in place.

Unfortunately, when they deactivate the account, they lose any ability to communicate with the customer. Anyone who emails claiming to be the customer may or may not be the same person.

That works both ways - as well as not being able to get the customers permission to keep the data, they also would not be able to get the customers request to delete the data if they were to keep it 'just in case'.

Until they have built some functionality to communicate with the customer behind a suspended account, they have to delete the data within 30 days.

Even building functionality to communicate with the person behind a suspended account is hard...

For example, if the login requires a password and SMS 2fa, but the SMS is to a Google Fi phone number, then you're going to have to also enable the Google Fi account for that user to be able to login. Or what if they need to use email password resets to another suspended account? (remember you will probably suspend all accounts of a user at once if you suspect something illegal).

Making a special 'chat' tool inside the login flow that only requires a single factor (the account password) which allows presenting evidence to a google rep seems like the obvious solution (or requesting the account suspension be delayed, or that the data be deleted).

Building and staffing that sounds like quite a lot of work though, which would be hard to justify for the tiny number of impacted accounts and zero revenue potential.

You make it sound like this is inevitable.

It isn't as far as I can see.

IMO it is just Google trampling over peoples rights while trying to use a weird reading of GDPR as excuse.

I'm so ready for EU to start punishing more of the blatant abuse of GDPR that we see, especially by the big players in the market.