News for hackers. Stuff that matters.
Re:Apache Pulsar is an open-source distributed pub-sub messa... by nostrebored on Thursday January 02, 2020 @06:20PM (#21941061)
Is there any reason you've decided not to use Dynamo to manage your materialized views?

Do you need to be able to generate a materialized view for a specific time window?

It feels weird to me to use your pub sub system to handle your persistent storage for views, but I am definitely missing context into pulsar and your use case

Re:Apache Pulsar is an open-source distributed pub-sub messa... by ryeguy on Thursday January 02, 2020 @10:37PM (#21942882)
Pulsar wouldn't be the thing that stores the view, it would be the canonical data store used to rebuild the views.

The idea is you have an event sourced view and can rebuilt it at any time by resetting the stream pointer to the beginning.

Re:Apache Pulsar is an open-source distributed pub-sub messa... by nostrebored on Saturday January 04, 2020 @06:10AM (#21954236)
Makes sense, so in this case realistically it would be replacing the data lake (likely S3) which currently in an AWS event driven architecture requires a lambda function to push to S3 and is vulnerable to network issues.

Seems like a cool use case!