roshan_naik

Not well known is that Heron's performance claims in the "flying faster" blog and SIGMOD paper were made by comparing against a fork of an older (pre apache) version of Storm even though much more performant version was already out.

IMO (biased?) Heron makes surprisingly poor design choices for their key distinguishing features like threading model and backpressure. And it shows when you benchmark it. More specifics: https://github.com/apache/storm/pull/2241#issuecomment-31787...

In the critical path many things can matter that ordinarily don't. When working on perf improvements for Storm 1.0 release, profiling revealed hotspots in Clojure code. Adding type hints to avoid dynamic typing fixed some of these hotspots. In other places, Clojure primitives showed up as a bottleneck. These required rewriting some code in Java and invoking it from Clojure.

This was mentioned briefly here: https://hortonworks.com/blog/microbenchmarking-storm-1-0-per...

The Clojure based core has served the community well for some time. It was still powered by Disruptor (a fast Java messaging library). So there was always some back and forth between Java and Clojure.

For Storm 2.0, the Java rewrite happened first. The re-architecture work might have not happened without it (at least for the foreseeable near future)... primarily due to Java expertise being more easily available.