Re:Apache Kafka and GDPR compliance
by nerpderp83 on Monday December 04, 2017 @04:28PM
(#15847674)
Encrypt with a user specific key when the data enters the log. You can effectively delete all the user specific data by throwing the key away. No tracking down files or reprocessing necessary.
Today’s encryption is strong. Who knows about tomorrow?
Is that acceptable within GDPR requirements?
Re:Apache Kafka and GDPR compliance
by nerpderp83 on Monday December 04, 2017 @05:24PM
(#15848120)
https://en.wikipedia.org/wiki/Crypto-shredding
https://law.stackexchange.com/questions/23375/gdpr-general-d...
I believe it would be, but IANAL.
The GDPR contains exceptions when deleting individual user data is infeasible or if the data in question is a backup, in which case you must only keep a log somewhere so you don't forget to delete again.
If we think around the options, you can have either:
i) eventual deletion (log retention policy) ii) compacted topics (and push null values) iii) expensive re-processing of the entire log iv) expensive segment re-write operation
with each option bringing in a new set of challenges