wording like this is really tricky (intentionally so!), it exploits our naive notion of digital information as files. One aspect is the machine learning you mention but basically they can also create a new "file" from your image with metadata about your image. Metadata rich enough to render the original image unnecessary. There is no way out of this relatiohship with FB, however they formulate words to comfort you, there is always a different unexpected side to what they are saying..

The more data they keep about these images that they are promising to delete, the more they would make themselves liable to be sued for privacy violation (especially in the EU).

It is a blurred line, but one that most rational companies (=not Uber) would prefer to avoid in the first place by only storing data that is necessary for their main functionality.

Before the EU sues there will be warnings and talks, plenty of time to collect a lot of faces.

But that is going to change in the new year with the new laws. with up to 2% of their global revenue in fines. And the fines IIRC can be issued backwards as well.

What is Facebook's "main functionality"?

Monetizing on people’s relationships and social addiction, collecting and providing detailed data on all user activities, sentiments and connections, selling sets of such data to ...?

Advertisers. If you've ever bought an ad on facebook and see you can pick your target audience by which types of food they shop for or their income range from 10k increments alongside thousands of other filters.... fuck man they have a helluva ad platform

Not facebook's main functionality. The feature's functionality.

e.g if you are building a captcha test feature, it should not keep more data than is needed for it to function.

This is the policy where I work at least, and I believe this is shared by most privacy-conscious companies (which facebook is... as far as the legal definition of privacy is concerned :)).

Oddly reminds me of the scene in Ex Machina where the scientists says he just turned on all phone's camera to mine face expression data.

... and the new iphone also wants to "set up FaceID", and Samsung's S8+ literally says "Face Recognition: Register your face." / "Fingerprint Scanner: Add your fingerprints." / "Iris Scanner: Register your irises".

FaceID is unique in the sense that it's supposed to be on-device only in the form of an ASIC coprocessor.

It's also something directly related to a face. "Want to unlock your phone by looking at it? It needs to know your face". But "Want to post a meme to Facebook? It needs your face" feels quite detached.

You can't tell with closed source. Anyway, what about "iCloud backup"?

Apple doesn’t backup TouchID or FaceID data to the cloud (just as they don’t backup your iOS device passcode). You need to set up each device independently.

https://images.apple.com/ca/fr/business-docs/FaceID_Security...

> Apple doesn’t backup TouchID or FaceID data to the cloud

We only have Apple's word to go by.

No, we don't.

https://hackaday.com/2017/08/18/apples-secure-enclave-proces...

Yeah it may be a binary dump, but it's now decrypted and understandable.

Serious question: can its firmware be upgraded through future device updates?

I THINK so.

https://www.theiphonewiki.com/wiki/Bootchain

https://www.theiphonewiki.com/wiki/Bootrom

https://www.theiphonewiki.com/wiki/25C3_presentation_%22Hack... (^F "The first piece" [of code])

https://www.cse.wustl.edu/~jain/cse571-14/ftp/ios_security/i...

The only part that is fundamentally read-only is the bootrom/SecureROM.

...Which is why https://ramtin-amin.fr/#nvmepcie and https://ramtin-amin.fr/#nvmedma are among my favorite articles ever. (iPhone 6, but still.)

Well, that and the word of numerous security researchers whose job it is to make sure that Apple is true to their word.

I'm pretty sure Facebook already has plenty of training data of faces

Excuse me. The issue is biometrics. "We'll delete the picture" =/= "we'll delete the control points and features extracted from your pics".

Under the EU GDPR (eg https://www.eugdpr.org/) it appears that they'll need explicit consent to retain the data (all of it, anything related and traceable to the person) and users will both have a right to view it and to have it deleted.

That only applies to Europe. The question is whether the data is valuable enough to Facebook to spend the time to add `if (europe)` to their code.

It applies to Europeans, even if they're not in Europe, and even if they don't identify themselves as being in Europe.

How exactly does the EU propose to apply its laws to people dealing with Europeans outside the EU?

Being European doesn't automatically subject everyone you interact with to EU laws, especially while you're outside the EU.

> How exactly does the EU propose to apply its laws to people dealing with Europeans outside the EU?

The EU can seize hard- and software of Facebook on their EU servers, the EU can seize Facebooks cash holdings in Ireland, the EU can seize (and sell off) Facebook’s patents held in the EU.

Stuff like this has happened before, and european governments have previously even seized airplanes from airlines right after landing to pay customers for a refund for a flight that the airline refused to pay.

This can, and absolutely will happen.

USA appear to have pushed their laws outside their natural jurisdiction ....

I think in practice it will mean Facebook, say, can't suddenly breach the normal EU regulations just because you went on a trip outside the EU.

To correct your correction I'd like to add that far from every European country is a member of the EU.

He is saying that that law applies to EU citizens even when they are not located in the EU. Much more difficult then determining which country the user is connecting from.

Not sure if this is the law though

And UK citizens, even if Brexit happens.

Or indeed care about the law enough to bother before they're fined for doing it. After all you'll need to prove they didn't delete the data first!

The GDPR penalties can reach 4% global (i.e. sum all related companies) annual turnover per infraction. If anyone ever found out Facebook had a widespread violation, Facebook would be likely be dissolved.

Oh boy would I be thrilled if that happens.

Nothing of value will really be lost, replacements would spring up super quickly and I'd like to think that we would have learned enough about the downsides of the current social-network systems for the next generation to avoid the pitfalls and hopefully make something much better.

I wish US corporate laws had teeth, it's only the criminal ones that apply to every day citizens that do.

Are there any instances of otherwise financially sound major global corporations being dissolved as a result of EU fines / regulatory action? I suspect if the rubber hits the road in this situation, FB would prevail.

Not dissolved, but the EU have shown themselves to be willing to tackle the tech behemoths, as Google recently found with their EUR2.4Bn fine https://www.theguardian.com/business/2017/jun/27/google-brac...

That's insane. What's a "single" infraction? It doesn't distinguish between revenue and profit? 4% of global revenue could be more than the total of all profit and salaries.

> 4% of global revenue could be more than the total of all profit and salaries.

That’s the point. Google recently came of comparatively well with a 4 billion EUR fine, but 2 more fines of the same height for Google are expected.

The only way to force a company to follow a law is to make infringing the law more expensive than any profit you could ever dream of making from it.

"Profit" can be kept at zero essentially forever. Revenue is much harder to hide.

> What's a "single" infraction?

There are lots of things that constitute an infraction: the GDPR aims to broadly protect EU citizen rights. Some infractions can be per individual harmed.

Then they better do their best not to infringe the privacy rights of EU citizens then, eh?

Yes.

And European courts consider intent carefully. Mere aggregation of personal data isn’t necessarily a violation, but using it commercially almost certainly is.

Someone in Europe should make a privacy complaint just to keep them on their toes.

You mean to step on their toes.

Yes, I guess it depends on what "picture" covers legally. Is a bijectively transformed version of your picture the same "picture"? How about a degraded version? How about a 3D model of your face?

Exactly. "We don't store your picture, we store an abstracted entity that allows us to recognize your picture in the future." The latter part is what people are concerned about anyway - the abstracted representation of "their face", not some particular photograph.

What do you mean by "control points"?

I think he means features like the corners of your mouth and eyes, points along the shape of your chin, etc.

Is there any precedent for recovering individual training items from a model trained on 100m+ examples?

What recovery? You're uploading your pic via your account. It is already associated with you.

As I understood it you are making the argument that they will have information about you from uploading this picture even if they delete the actual picture. I'm genuinely asking if it is possible to recover an individual training example from a model which is shown vast amounts of data. I'm only aware of being able to recover stuff like this - https://ars.els-cdn.com/content/image/1-s2.0-S08936080173020... which is hardly PII

The goal is to be able to identify you, not recover an image of you.

This is just as much PII as your fingerprints. Fingerprint based devices store metadata not images. I can't look at a fingerprint and connect it to a person, but a machine can. Similarly the output of a NN designed to recognize people will be useful for validation. Otherwise biometrics would not work.

Remember if they are using your photo as training data the NN is going to learn you vs someone else or it's useless. If they can find you in any photo that's as effective as handing a person the uploaded photo and giving the same task.

I understood your question. What I am pointing out is that when you submit an artifact to FB, it is a trivial matter for them to store as 'extended-attributes' your facial biometrics. This has nothing to do with the ML pipeline that munches on data in the aggregate.

As to whether it is possible to extract such information, per my understanding the internals of ML pipelines are rather opaque, so it would be non-trivial, if not impossible. I have no idea either way.

I think there are two separate questions being raised. My understanding (EDIT: my understanding of the concern being raised in this discussion) was not that they would try to recover individual readings from a trained model, but that they would run a model on your face to extract features, then save an array of features that defines your face. Only those features are needed to identify your face - after data extraction the photo is no longer needed.

This article talks about how those data points are extracted using a different method.

https://medium.com/@ageitgey/machine-learning-is-fun-part-4-...

It is not about being able to recover, rather about being able to identify your face on other photos.

You are excused. Facebook has lots of photos of my face already, will one more make a difference?

Not everyone uploads their face to facebook over and over and over again.

Or at all.

Don’t worry we’ve got you covered! (Tags Dr Queer.)

But most do.

Most humans don’t even have a facebook account. Did you mean “most people in my observable social circle and subculure”? Just because many of my generation willingly give their identity away does not mean this behavior is normal or that this is an excusable form of exploitation.

That is correct, currently only about one third of all humans over 13 are active users of Facebook.