Re:Outgrowing Apache Storm: why we built in-house distribute...
by rayscondition on Thursday October 01, 2015 @03:33PM
(#10314044)
Hi, I'm the author of the blog post and wrote SuperChief. We chose Java 8 because it's the language we're most comfortable with on the data team and find it easiest to reason about. We also reused some of the existing bolt code we wrote for Storm to run the time series aggregations. It was already written in Java so we just had to make it thread safe. I suspect you could get similar performance with other jvm based languages.
I was interested in looking at Spark for this for a bit but coming from Storm we decided we wanted something stripped down that was more purpose built.
Re:Outgrowing Apache Storm: why we built in-house distribute...
by meatysnapper on Thursday October 01, 2015 @03:36PM
(#10314062)
If I remember right, the team that wrote Heron was most comfortable with C++, and that was a major reason in choosing it.
IIRC, Twitter's Storm replacement, Heron, is written in C++ -- i.e. they didn't go with Spark/Scala, which, given that Twitter is probably the largest Scala shop in the world, speaks volumes about the volume of data these sytems need handle (read: Spark is far from slow).