News for hackers. Stuff that matters.

A comment on Storm's first birthday.

Re:Storm's first birthday by newobj on Wednesday September 19, 2012 @09:54PM (#4546584)
What I'd like to understand at a glance about Storm is durability, idempotency, and specifically those two things in the face of all your standard failure modes.

Is Storm something that is going to get 99.9% of my data there which is good enough for some side-channel processing like ad targeting, or is it something that's D durable in the strongest sense of the word? If I'm ultimately delivering data to something that is 5 9's durable, or better yet S3 10-11 9's durable, is Storm going to be my "low point" of durability?

[ Parent | on HN ]
Re:Storm's first birthday by djb_hackernews on Wednesday September 19, 2012 @10:18PM (#4546646)
Read the wiki:

"Guarantees no data loss: A realtime system must have strong guarantees about data being successfully processed. A system that drops data has a very limited set of use cases. Storm guarantees that every message will be processed, and this is in direct contrast with other systems like S4."

https://github.com/nathanmarz/storm/wiki/Rationale

[ Parent | on HN ]
Re:Storm's first birthday by mattmcknight on Wednesday September 19, 2012 @11:38PM (#4546815)
Storm supports transactional topologies that can provide durability. They are dependent on a spout that is repeatable- where you can request the failed input again. https://github.com/nathanmarz/storm/wiki/Transactional-topol...
[ Parent | on HN ]
Re:Storm's first birthday by nathanmarz on Thursday September 20, 2012 @12:51AM (#4547011)
Actually transactional topologies are superseded by Trident, which is much easier to use:

https://github.com/nathanmarz/storm/wiki/Trident-tutorial

Here's an explanation on how Trident achieves fully fault-tolerant, idempotent semantics:

https://github.com/nathanmarz/storm/wiki/Trident-state

[ Parent | on HN ]