Agreed. Our current implementation does not work when there are multiple readers for the same event stream and we need to track per-reader watermarks. We have a very simple model where one reader reads from DB and distributes the work to multiple workers (e.g. network writers) which in turn update the job status.

Multiple writers should work though. StoreJob() should handle that.

I missed the logging to RAM part. Yes, we always wanted to do that but haven't gotten to that yet. Right now, all events are moved through the DB - between gateway and processor and then router. Hence, we poll the table constantly.

Would love if you join our discord channel https://discordapp.com/channels/625629179697692673/625629179.... Slightly easier to have technical discussion there :)