Show HN: Sandglass – Distributed, scalable, persistent time-sorted message queue

Re:Show HN: Sandglass – Distributed, scalable, persistent ti... by menacingly on Monday December 11, 2017 @10:19AM (#15897502)

It triggers my eyebrow when a highly available message queue offers either exactly-once delivery or reliable time sorting.

I'm not saying I wouldn't give this project a closer look, but I would much rather a product make it painfully obvious what compromises were made to offer its availability.

My instinct is: either you need strict constraints on ordering and delivery, in which case you use rabbit, or you need at-least-once-no-matter-what semantics, in which case you use something else and make your app less fragile.

[ Parent | on HN ]

Re:Show HN: Sandglass – Distributed, scalable, persistent ti... by eternalban on Monday December 11, 2017 @02:22PM (#15899518)

Haven't looked but the ordering has to be per partition.

> something else

https://www.confluent.io/blog/exactly-once-semantics-are-pos...

[ Parent | on HN ]

Re:Show HN: Sandglass – Distributed, scalable, persistent ti... by meebs on Monday December 11, 2017 @11:14AM (#15897951)

Hey, what would "something else" be in this scenario?

[ Parent | on HN ]

Re:Show HN: Sandglass – Distributed, scalable, persistent ti... by menacingly on Monday December 11, 2017 @01:22PM (#15899045)

Usually it would advertise "at-least-once" somewhere prominently, like SQS, NSQ, etc.

It's really hard to be extremely available and to also guarantee there won't be dupes or out-of-order messages. Like databases, someone saying they offer both is usually hiding a trade-off.

EDIT: I think here "really hard" is a polite way of saying "impossible"

[ Parent | on HN ]

Re:Show HN: Sandglass – Distributed, scalable, persistent ti... by markbnj on Monday December 11, 2017 @02:06PM (#15899394)

Perhaps redis streams http://antirez.com/news/114

[ Parent | on HN ]

Re:Show HN: Sandglass – Distributed, scalable, persistent ti... by dkersten on Monday December 11, 2017 @12:45PM (#15898690)

Kafka perhaps?

[ Parent | on HN ]

Re:Show HN: Sandglass – Distributed, scalable, persistent ti... by lifeisstillgood on Monday December 11, 2017 @06:06PM (#15901565)

do you know a good primer on the basics behind all these issues.

i can sorta see how i would build a message queue that ordered by time, but lost messages would be a problem and so forth - just to get n idea of the trade offs

[ Parent | on HN ]