News for hackers. Stuff that matters.
Re:Kafka, GDPR and Event Sourcing by BillinghamJ on Thursday April 12, 2018 @09:08PM (#16826330)
Crypto-shredding, as it is known, it is already well established as a valid process and covers the requirements.

The point is to make the data unusable. Obviously when you delete a file from your recycle bin, the data is all still there, but it's acceptable under EU data protection reqs.

Re:Kafka, GDPR and Event Sourcing by TheForumTroll on Friday April 13, 2018 @07:10AM (#16828848)
>but it's acceptable under EU data protection reqs

No it isn't. This is what got Facebook in hot waters awhile back. Marking a picture as deleted isn't the same as deleting it. Some other mechanism could easily point to where the data is actually located and not just its descriptor.

Re:Kafka, GDPR and Event Sourcing by BillinghamJ on Friday April 13, 2018 @12:14PM (#16831375)
Yes, it is. The legal definition of deletion under EU data protection laws is to make the data unusable and inaccessible. Marking for deletion (as deleting from a recycling bin does) is sufficient under those rules.

Deleting the image ID and leaving the content on a CDN is not acceptable - but that's not what I said.

Re:Kafka, GDPR and Event Sourcing by sandGorgon on Thursday April 12, 2018 @10:01PM (#16826605)
How does this work in practice? If each customers data has a unique encryption key, then how do you do SQL joins ?
Re:Kafka, GDPR and Event Sourcing by whalesalad on Thursday April 12, 2018 @10:13PM (#16826655)
It’s pretty straightforward: you don’t encrypt the data necessary for a join.

Data for a join, ie foreign keys, are used to inform your structure. They aren’t really data a person owns... they just describe how that data (theirs) fits into your database (yours).

Re:Kafka, GDPR and Event Sourcing by PeterisP on Friday April 13, 2018 @02:42AM (#16827606)
You'd still need to decrypt the whole table to do e.g. aggregation.

For example, a customer's address is private information; statistics about how many customers come from each town/region is not, but calculating a simple "select town, count(*) from x group by town" requires decrypting every address.

Re:Kafka, GDPR and Event Sourcing by leifg on Friday April 13, 2018 @03:24AM (#16827788)
you can store whatever you want unencrypted on the read model. That‘s the place where you do joins/aggregation anyway.
Re:Kafka, GDPR and Event Sourcing by DelightOne on Friday April 13, 2018 @07:38AM (#16828991)
> you can store whatever you want unencrypted on the read model.

So to recap correct me if I’m wrong:

   You store everything encrypted.
   You have an api on it which hides the encryption and shows everything unencrypted.
   On the unencrypted api you do joins/aggregations.
   RightToBeForgotten then means delete its decryption key and remove it from the in-memory api.
Right?
Re:Kafka, GDPR and Event Sourcing by xyzzy_plugh on Thursday April 12, 2018 @11:35PM (#16826958)
You can always join on decrypt(key, a.column) = decrypt(key,b.column).