Re:Apache Pulsar is an open-source distributed pub-sub messa...
by addisonj on Thursday January 02, 2020 @07:05PM
(#21941435)
Pulsar docs indicate "millions" of topics but IDK what 50 million would look like but from what I know I would be a bit nervous about it :)
It might be worth chatting with Pulsar devs on their slack community (https://apache-pulsar.herokuapp.com/).
Most commonly what I hear people doing for this is either one of two approaches (or a combination of both): - encrypt the user data and delete the key, eventually the user data will get removed - regularly compact the topic (pulsar has a compaction feature) and write in a tombstone record which will remove any user data after compaction
For GDPR a lot of us has to do exportable 'user activity'. Can you in theory have a topic/user ( we had like 50 million users) and publish any user activity to that topic?