News for hackers. Stuff that matters.
Re:Kafka, GDPR and Event Sourcing by noir_lord on Thursday April 12, 2018 @07:54PM (#16825985)
The forget the encryption key approach is clever.
Re:Kafka, GDPR and Event Sourcing by jordibunster on Thursday April 12, 2018 @08:18PM (#16826099)
I used to think so too (that's what iPhones and the like do on "erase", btw) but doesn't that just push the problem into the future, when computers can more easily crack today's encryption?
Re:Kafka, GDPR and Event Sourcing by xoa on Thursday April 12, 2018 @09:13PM (#16826355)
>the future, when computers can more easily crack today's encryption

No. Don't confuse symmetric with asymmetric (current public key) encryption. They aren't subject to the same potential attacks. Even with a theoretical fully scalable general purpose quantum computer, the best quantum attack vs a symmetric cypher is brute forcing with Grover's Algorithm, which provides a quadratic rather then exponential speed up. Ie., a n-bit key could be attacked in around 2^(n/2). This is trivially countered by doubling key length, a 256-bit key would still take 2^128 which would still be effectively impossible, and a 512-bit key would take 2^256. There is no future with any foreseen technology that would be able to brute force that, so when it comes to AES and the like using at least 256-bit keys it can be reasonably assumed that destroying the key means the data is lost (anything legacy running off 128-bit is reasonable to watch out for though, 2^64 is potentially tractable).

Present asymmetric crypto systems can theoretically [1] be attacked with Shor's Algorithm, which may be what you're kind of thinking of if you've heard about "today's encryption getting cracked" in the general media or scifi. And that would in fact be a big deal, it covers how most data is moved around in communications and the Internet at present. But QC isn't magic, and it doesn't just break anything. FDE and the like that just use symmetric crypto are safe.

1: "Theoretically" because that's if (big if) an ideal quantum computer that could be scaled to a sufficient number of qubits is created.

Re:Kafka, GDPR and Event Sourcing by codetrotter on Friday April 13, 2018 @08:06AM (#16829156)
> There is no future with any foreseen technology that would be able to brute force that, so when it comes to AES and the like using at least 256-bit keys it can be reasonably assumed that destroying the key means the data is lost (anything legacy running off 128-bit is reasonable to watch out for though, 2^64 is potentially tractable).

But we can’t know for sure that AES or any other encryption algorithm doesn’t have some as-of-yet-unknown fatal flaw that would make it breakable in some way not necessarily even having anything to do with quantum computers?

Re:Kafka, GDPR and Event Sourcing by gizzlon on Friday April 13, 2018 @08:34AM (#16829353)
Of course not. But now you're getting philosophical :) Can you really now anything?
Re:Kafka, GDPR and Event Sourcing by codetrotter on Friday April 13, 2018 @11:50AM (#16831158)
Ha ha yeah that is true. My point though was that it’s important to keep in mind if we decide to use throwing away the encryption key as or way of protecting the data.
Re:Kafka, GDPR and Event Sourcing by raarts on Friday April 13, 2018 @09:25AM (#16829725)
> Don't confuse symmetric with asymmetric (current public key) encryption

After reading this multiple times, and looking up QC (Quantum Computing) and FDE (Full Disk Encryption), I got the following out of it:

- symmetric encryption is safe for AES-256 and up - asymmetric encryption isn't getting cracked because QC isn't magic

Is this a correct TL;DR ?

Re:Kafka, GDPR and Event Sourcing by prrls on Saturday April 14, 2018 @06:00AM (#16836683)
The computations used to do symmetric and asymmetric encryptions are completely different. Asymmetric encryption is just about factoring out big numbers into primes, if we simplify things a bit. And modern computers aren't very good at it, but Quantum Computer happens to be, and can break it. Read a bit about Shor's Algorithm[0].

On the other hand, symmetric encryption can be seen as a super convoluted and costly shift cipher. And it seems that Quantum Computing does not help much with dumb and costly mathematics like this.

[0]: https://en.wikipedia.org/wiki/Shor%27s_algorithm

Re:Kafka, GDPR and Event Sourcing by Terr_ on Thursday April 12, 2018 @09:19PM (#16826386)
It'll be fine as long as the decryption time-horizon is beyond the lifetime of the company, the user, and the regulatory-regime :p
Re:Kafka, GDPR and Event Sourcing by theyinwhy on Friday April 13, 2018 @01:04AM (#16827240)
Not even this as it could have negative impact on the next at least 2 generations depending on politics in the future. Some people judge you based on who your ancestors were and what they did. There are countries today where you have to fear for your life if your father was gay, for example.

As we cannot foresee the future, neither politics nor any future decyphering capacities, I highly doubt deleting the key is a viable option.

Note: Edited for clarity.

Re:Kafka, GDPR and Event Sourcing by evrydayhustling on Thursday April 12, 2018 @08:50PM (#16826240)
Yes very cool! But I wonder if courts will decide that throwing away the key to the lockbox is the same as burning the contents.
Re:Kafka, GDPR and Event Sourcing by BillinghamJ on Thursday April 12, 2018 @09:08PM (#16826330)
Crypto-shredding, as it is known, it is already well established as a valid process and covers the requirements.

The point is to make the data unusable. Obviously when you delete a file from your recycle bin, the data is all still there, but it's acceptable under EU data protection reqs.

Re:Kafka, GDPR and Event Sourcing by TheForumTroll on Friday April 13, 2018 @07:10AM (#16828848)
>but it's acceptable under EU data protection reqs

No it isn't. This is what got Facebook in hot waters awhile back. Marking a picture as deleted isn't the same as deleting it. Some other mechanism could easily point to where the data is actually located and not just its descriptor.

Re:Kafka, GDPR and Event Sourcing by BillinghamJ on Friday April 13, 2018 @12:14PM (#16831375)
Yes, it is. The legal definition of deletion under EU data protection laws is to make the data unusable and inaccessible. Marking for deletion (as deleting from a recycling bin does) is sufficient under those rules.

Deleting the image ID and leaving the content on a CDN is not acceptable - but that's not what I said.

Re:Kafka, GDPR and Event Sourcing by sandGorgon on Thursday April 12, 2018 @10:01PM (#16826605)
How does this work in practice? If each customers data has a unique encryption key, then how do you do SQL joins ?
Re:Kafka, GDPR and Event Sourcing by whalesalad on Thursday April 12, 2018 @10:13PM (#16826655)
It’s pretty straightforward: you don’t encrypt the data necessary for a join.

Data for a join, ie foreign keys, are used to inform your structure. They aren’t really data a person owns... they just describe how that data (theirs) fits into your database (yours).

Re:Kafka, GDPR and Event Sourcing by PeterisP on Friday April 13, 2018 @02:42AM (#16827606)
You'd still need to decrypt the whole table to do e.g. aggregation.

For example, a customer's address is private information; statistics about how many customers come from each town/region is not, but calculating a simple "select town, count(*) from x group by town" requires decrypting every address.

Re:Kafka, GDPR and Event Sourcing by leifg on Friday April 13, 2018 @03:24AM (#16827788)
you can store whatever you want unencrypted on the read model. That‘s the place where you do joins/aggregation anyway.
Re:Kafka, GDPR and Event Sourcing by DelightOne on Friday April 13, 2018 @07:38AM (#16828991)
> you can store whatever you want unencrypted on the read model.

So to recap correct me if I’m wrong:

   You store everything encrypted.
   You have an api on it which hides the encryption and shows everything unencrypted.
   On the unencrypted api you do joins/aggregations.
   RightToBeForgotten then means delete its decryption key and remove it from the in-memory api.
Right?
Re:Kafka, GDPR and Event Sourcing by xyzzy_plugh on Thursday April 12, 2018 @11:35PM (#16826958)
You can always join on decrypt(key, a.column) = decrypt(key,b.column).
Re:Kafka, GDPR and Event Sourcing by freedomben on Thursday April 12, 2018 @11:32PM (#16826954)
This is clever, but it requires strict following of best practices (nobody has copies of production data, and backups and the like are guarded with proper access controls. Keys are never written to log files, etc).

Make sure to get any DB backups as well, otherwise the key could still be out there somewhere. If you store the key under version control as well (bad practice but we've all seen it done) you'll need to rewrite git history to make sure the key isn't recoverable at all.

There could be an infinite number of ways a key could unintentionally persist, so something to keep in mind with this approach. Good reason to be disciplined with keys and other secrets to make sure they aren't leaving the environment.

Re:Kafka, GDPR and Event Sourcing by nbevans on Friday April 13, 2018 @04:37AM (#16828081)
Why is storing junk data for all eternity considered clever?
Re:Kafka, GDPR and Event Sourcing by raarts on Friday April 13, 2018 @09:05AM (#16829579)
Because it beats getting a multimillion dollar fine from the EU?