What is the difference between Spark and Storm? They both seem like "realtime compute engines"
*edit - from what I can see Spark is a replacement for hadoop (offline jobs), where Storm deals with online stream processing
Storm is generally more of a dataflow "per event" real/near time computation system (with each event flowing through N spouts and bolts) whereas Spark is more of an in-memory data processing system (with Spark streaming being the "equivalent" to the storm system).
By comparison, I'm trying (and failing) to work with RDDs of 100+ billion elements.