Re:History of Apache Storm and Lessons Learned
by skybrian on Monday October 06, 2014 @03:09PM
(#8417602)
How does it compare to Apache Kafka? (I haven't used either.)
[ Parent
| on HN ]
Re:History of Apache Storm and Lessons Learned
by akbar501 on Monday October 06, 2014 @03:22PM
(#8417686)
Kafka is a high performance, persistent message queue.
Storm is distributed stream processing.
The two work together very well. Often, a server will publish messages to Kafka (or another queue), which are then read and processed by Storm.
Re:History of Apache Storm and Lessons Learned
by throwaway1979 on Monday October 06, 2014 @04:39PM
(#8418155)
Interesting question and response. I haven't used either but read about both systems independently. A bit more googling and I found:
http://www.slideshare.net/gschmutz/kafka-andstromeventproces...
Slides 42 and 43 describe an architecture with all three ... hadoop, storm and kafka. Seems like a beast and I don't have applications but pretty cool combo of technologies.
Re:History of Apache Storm and Lessons Learned
by erichmond on Monday October 06, 2014 @04:58PM
(#8418264)
http://manning.com/marz/ - this is often the trifecta used in the "lambda architecture", coined by nathan marz. This is also the system they built @ backtype to do their real-time analytics system.
Re:History of Apache Storm and Lessons Learned
by phunge on Monday October 06, 2014 @06:25PM
(#8418695)
Also worth checking out http://radar.oreilly.com/2014/07/questioning-the-lambda-arch... from Jay Kreps, who's part of the Kafka braintrust.
Re:History of Apache Storm and Lessons Learned
by pixelmonkey on Monday October 06, 2014 @06:50PM
(#8418789)
Complementary technologies, often used together. In fact, in Storm 0.9.x, Storm actually bundles a JVM implementation of a Storm Spout that plugs directly into Kafka:
https://github.com/apache/storm/tree/master/external/storm-k...
Re:History of Apache Storm and Lessons Learned
by schultz9999 on Monday October 06, 2014 @10:23PM
(#8419421)
In no way -- apples vs. oranges. Kafka is queue, Storm is processing engine. A better question how it's compared to other processing engines like Spark Streaming, Samza, etc. But just Google it :)