Re:Kafka, GDPR and Event Sourcing
by sandGorgon on Thursday April 12, 2018 @10:01PM
(#16826605)
How does this work in practice? If each customers data has a unique encryption key, then how do you do SQL joins ?
[ Parent
| on HN ]
Re:Kafka, GDPR and Event Sourcing
by whalesalad on Thursday April 12, 2018 @10:13PM
(#16826655)
It’s pretty straightforward: you don’t encrypt the data necessary for a join.
Data for a join, ie foreign keys, are used to inform your structure. They aren’t really data a person owns... they just describe how that data (theirs) fits into your database (yours).
You'd still need to decrypt the whole table to do e.g. aggregation.
For example, a customer's address is private information; statistics about how many customers come from each town/region is not, but calculating a simple "select town, count(*) from x group by town" requires decrypting every address.
you can store whatever you want unencrypted on the read model. That‘s the place where you do joins/aggregation anyway.
Re:Kafka, GDPR and Event Sourcing
by DelightOne on Friday April 13, 2018 @07:38AM
(#16828991)
> you can store whatever you want unencrypted on the read model.
So to recap correct me if I’m wrong:
You store everything encrypted.
You have an api on it which hides the encryption and shows everything unencrypted.
On the unencrypted api you do joins/aggregations.
RightToBeForgotten then means delete its decryption key and remove it from the in-memory api.
Right?
Re:Kafka, GDPR and Event Sourcing
by xyzzy_plugh on Thursday April 12, 2018 @11:35PM
(#16826958)
You can always join on decrypt(key, a.column) = decrypt(key,b.column).