News for hackers. Stuff that matters.
Re:Show HN: DriftDB is an open source WebSocket backend for ... by paulgb on Monday February 06, 2023 @10:58AM (#34679294)
Cool, looking forward to the example :)

I actually didn’t realize that the expected last sequence could be combined with nats-rollup. In the example above (as I understand it) if we added that to the snapshot, NATS would throw out the snapshot, so the log would still be correct, but the work of snapshotting it and sending it over the wire would be lost. If messages were frequent enough, and/or snapshots took a while to compute/transfer, you might never have a roll-up succeed.

Our approach is that a roll-up will always succeed, we just preserve any messages in the stream with a sequence number greater than the one provided.

(I’m enjoying it too, and am a user of NATS so I’m happy to learn things I didn’t know about it :)

Re:Show HN: DriftDB is an open source WebSocket backend for ... by bruth on Monday February 06, 2023 @02:30PM (#34682859)
> If messages were frequent enough, and/or snapshots took a while to compute/transfer, you might never have a roll-up succeed.

Yes, good point. The "snapshotting up to a lagging sequence" could be achieved with two separate subjects to reduce contention, but is a bit more work.

It sounds like, in Drift's case, the snapshot effectively brings up the tail (snapshot), but the head can still be appended to with new events.

Re:Show HN: DriftDB is an open source WebSocket backend for ... by paulgb on Monday February 06, 2023 @03:01PM (#34683426)
> It sounds like, in Drift's case, the snapshot effectively brings up the tail (snapshot), but the head can still be appended to with new events.

Yep, exactly. It’s a subtle feature but it makes it possible for multiple clients to attempt to compact without worrying about races.