News for hackers. Stuff that matters.
Re:Sirius: A distributed system library from Comcast by agibsonccc on Wednesday April 23, 2014 @01:28PM (#7635472)
Hazelcast is a favorite of mine: http://hazelcast.org
Re:Sirius: A distributed system library from Comcast by bri3d on Wednesday April 23, 2014 @01:36PM (#7635520)
On cursory first glance Sirius looks more advanced than Hazelcast: Sirius seems to provide substantially more flexible cluster topologies along with "real" consistency behavior (via Paxos). Plus, Sirius seems to come with a built-in persistence system with at least some thought applied to it, while last I checked Hazelcast left persistence entirely up to the library consumer.

As a disclaimer I haven't looked at the implementations of either, only the documentation (and I used Hazelcast in a project a while ago).

Re:Sirius: A distributed system library from Comcast by larsmak on Wednesday April 23, 2014 @03:10PM (#7636211)
Having used HazelCast extensively, and considering making some of my code Sirius-depentant, I can tell you that it is not more advanced. Sirius is basically just a distributed key-value store w/no partitioning, whereas HazelCast has a lot of abstractions built on top of the simpler principle[1]. In fact, it might be a bit overkill in some situations, and that's one of the reasons I will consider Sirius.

[1] http://www.hazelcast.org/docs/3.1/manual/html-single/

Re:Sirius: A distributed system library from Comcast by comcast-jonm on Wednesday April 23, 2014 @04:34PM (#7636716)
Our particular reference dataset (TV and movie data) had some application use cases that required custom data structures to get the in-memory performance we wanted. Sirius doesn't provide datastructures per se (and isn't really even a datastore itself)--it provides an eventually-consistent, replicated update stream that you can use to maintain your own datastructures. Having that developer freedom and control over the datastructures was an important design principle.
Re:Sirius: A distributed system library from Comcast by bri3d on Wednesday April 23, 2014 @03:31PM (#7636351)
Good point!

I agree and I think we were looking at different dimensions: the consistency/partition tolerance story (the "distributed system" bits) seem more thought out in Sirius while the variety of data structures and connectors look more built up in HazelCast.

This may be an issue of documentation as well: Because it's built more "vertically" with a large consumer-facing cross-section (lots of data structures stacked over each other), the HazelCast documentation seems to be very use-oriented while the Sirius documentation focuses on the fundamentals.

Re:Sirius: A distributed system library from Comcast by agibsonccc on Wednesday April 23, 2014 @04:06PM (#7636575)
Yeah, I definitely use hazelcast for the variety of data structures.

I personally use it for a variety of tasks including distributed locks, a task tracker (think like hadoop) and for phases. If you're curious, here's the impl I'm talking about:

https://github.com/agibsonccc/java-deeplearning/blob/master/...

This is to augment an akka clustering distributed run time.