We don't soft delete payloads at Raygun (https://raygun.com), for the very fact that typically if one of our customers wants to delete something it's because they might have sent something they don't want a third party to have. We have filters and other PII filtering tools etc, but it every now and then something might be sent by mistake.

Having said that, you'd be amazed how often folks ask for things to be undeleted (despite a big warning dialog).

Clearly developers pervasively believe soft deletes are occurring everywhere.

It isn’t that hard to combine soft deletes with delayed hard deletes: generate a new encryption key every day for “data deleted today”, and encrypt deleted data with it. After X days, destroy the decryption key.

If you use asymmetric encryption, you can keep the group of people who who can recover “deleted data” small. You could even have an independent party generate your encryption key pair, give you the encryption key, and your customer, on request, the decryption key (I think there is a business model for a non-profit here).

Instead of having a key that you delete (and also build non trivial infrastructure to support), why not delete the actual data?

Because the key is smaller, it is easier to make sure you deleted every copy of that key than that you deleted every copy of the data. The data also might be part of a larger backup that you would have to take apart and reassemble in order to delete the data, or might be in a place where doing that is costly (e.g. on Amazon Glacier)

It seems precisely as easy to make sure you've deleted every copy of the data as it is to make sure you've encrypted every copy of the data.

Edit: apologies, seems I read way too quickly! Thanks for pointing it out.

You seem to be commenting out of context:

> generate a new encryption key every day for “data deleted today”,

The question is not can we encrypt at storage. We’re now talking about encrypting as a soft-deletion method, which means we need to know everywhere the data is stored at deletion time, whether to delete it or to encrypt it with this new “deletion” key.

Thanks for raising that issue, I was somewhat confused by the mentioning of encryption as a soft-deletion method... it made precious little sense to me, but everybody seemed to go along with it and I thought I was missing something very fundamentally ’right’ about that idea. Turns out it’s not so.

Absolutely!

That's great you guys do that. But it cant be proven, why take your word for it?

Ultimately, its the trust that is ghe problem, and that is what needs to be removed eother through new technology or legislation or both.

and why offer the false sense of security?

if they upload a private key, and delete because they "don't want a third party to have". do you also guarantee it wasn't seen or cached anywhere else? I dont know the details of that product, but I usually treat anything uploaded even once as compromised from that point on.

This is the same argument people used to make for why it was fine for capabilities to be unrevocable--someone could have copied the data anyway (or whatever) so there was no point in revoking it. In reality, most of the time nobody but the host of a deleted item has access to the data, has a way to tie it to the originator, and has a motive to use it, especially without significant effort. Being able to delete things is a very important feature (not to mention a legal requirement in many countries!), and it's disturbing to me how many people seem to want to justify a world where every bit of data is saved, forever.

How do you handle:

1. Deletions from backups

2. Deleting material that has been deleted prior to the restoration of the backup?

> Not to alleviate facebook of blame, but who's to say data on almost every other social media service isn't also just flagged for deletion?

The word "delete" has a pretty clear definition to most users. Facebook is one of the most used pieces of software in the world. If FB is allowed to lie to its users, it would indeed give a pass to just about every social media service out there.

The reason Facebook is special, and deserves special scrutiny, is because of its power. If FB establishes a bad behavior, it will become the norm.

A more prudent question would be whether these tech companies should be reined in by federal privacy law. Should they be allowed to collect, trade and analyze private data on all of its users? Where do we draw the line in the sand, in terms of what's acceptable and not.

These are incredibly important questions. A related field would be the credit bureaus, such as Equifax. Global companies who store social security numbers and all other sorts of information. We need a national set of rules for these companies to follow.

Not keeping my hopes up, given our Congress is so dysfunctional these days.

I would think undelete also has a clear definition. Technical implementation is orthogonal to these kinds of definitions.

Does it make it ok for Facebook to do it just because similar other companies do it? I say no, all of them should delete something I say to delete. And "everyone does it" is makes it a bigger problem, not a smaller one.

A lot of the big agile companies are using event sourcing. So there isn't even a model to delete. It's all events with the models being created from a snapshot of events. The event stream is usually durable and lives forever.

https://martinfowler.com/eaaDev/EventSourcing.html

So with this type of system nothing is ever "deleted". It's just an event that something is deleted.

This is a common and very scalable system. You don't deal with models, you deal with events (and a model is a snapshot of events).

This is even an issue. Even other companies that aren't event sourcing, but traditional model architecture have backups. You ask something to be deleted and they might actually delete it, but what about last weeks backup? It's not deleted there.

User expectations about a deletion would probably be "make it as though this was never uploaded": no copy, no backup, no recoverable form whatsoever. And yet, if it was not deleted, and there was a problem requiring backups to be used, they would expect to never even know about it, just that all their data would remain.

It's very much against the rules in event sourced systems to change history. But maybe that just doesn't matter. If it means you can never meet a user expectation about privacy, I guess you could tell the user that everything persists indefinitely... or when something is deleted, go back to the upload event and remove it, rebuilding history with any event related to that uploaded item ignored. Putting the user above the "purity" of the software and creating potential problems elsewhere.

Even on backups in long term storage, there could be some process of creating new copies of the backups with any needed modifications on some kind of schedule, so deletions can propagate over time.

Ultimately the challenges here are financial. We could delete things thoroughly if we were willing to pay for the developer time and other resources needed to make it work.

I was thinking the other day that GDPR might make event sourcing problematic.

"You are strongly encouraged to backup a database before excising data."

You are supposed to delete old backups.

Why? Disk space is cheap.

Why would they? They implement their system the way they want to. Also, this is a completely logical way to deal with deletions. This is what I would do, (what I have done, when I created a simple CMS system). I don't want an endless quarrel with a customer, who "accidentally" deleted something and wants it back. I just turn the switch and it is back.

Like I said, no respect for user choices.

Why even have a delete button? Why lie to your customers about their privacy?

What does this have to do with privacy?

Do you mean the delete button is a lie? Why would it be a lie? Can you or someone else access the deleted video from Facebook.com? Or in another public way? Isn't it deleted from this point of view?

I am not defending Facebook in anyway. I just don't understand why is everybody surprised about these things. Do you think if you click delete on a video on youtube, then it physically deletes the video from all of its servers?

I recall the distinction being made very clear on LiveJournal between "deleted" content vs "purged". I would be very surprised if they were not being forthright about this. Of course this was 10 years ago, before the Russian ownership. So I do have reason to believe that not all companies act in deceitful ways when it comes to retention of user data.

Snapchat claims to. And after the trouble they once got into for not deleting media after it was viewed, I believe them.

https://www.snap.com/en-GB/privacy/privacy-policy/

Why would you believe them after they have lied to you once already?

Right, I wonder how much people trust American companies. Imagine a Chinese firm doing the same and how many will trust them again..

It is best to never have any Chinese company store your data. They are by law (and under severe penalties) required to make all data in their possession available to government officials at any time that it is requested. Dictatorships are like that.

Chinese firms are generally required under threat of arrest to store and/or transmit all data on users. A single Chinese firm failing to delete data would have little or no negative impact within China given that they’re probably already secretly required to do exactly this. Do you mean US customer impact?

This is why:

https://www.ftc.gov/news-events/press-releases/2014/12/ftc-a...

https://arstechnica.com/tech-policy/2014/12/snapchats-privac...

Nothing in those articles suggest that Snapchat purged user data.

> “resolved most of those concerns over the past year by improving the wording of our privacy policy, app description, and in-app just-in-time notifications.”

Snapchat just changed their messaging to quell user concerns. Once they have a critical mass of users, they are immune to disclosing that Snapchat messages are not truly ephemeral.

I understand your skepticism considering the behavior we've seen from some of these companies recently — but when their settlement with the FTC includes an independent company monitoring their handling of user privacy for 20 years, I think it's safe to trust them on this one.

>Snapchat servers are designed to automatically delete all Snaps after they’ve been viewed by all recipients

>Snapchat servers are designed to automatically delete all unopened Snaps after 30 days

https://support.snapchat.com/en-GB/a/when-are-snaps-chats-de...

Yeah i don't see much difference between this and hitting delete on a file in a local file system. The data itself still sits there until the sectors gets reclaimed, but there is no longer a file name or directory entry associated with them.