Makes total sense, especially the part "Superchief is not an off-the-shelf-framework that anyone will be able to drop-in to fulfill their stream processing requirements".

So, essentially, Storm solved a more generic problem than you needed it to solve. Superchief is a narrower system that more closely aligns with your problem domain, and this let you not only simplify a little but also gain efficiency. Makes sense!

(p.s. the reason I ask is because I am deeply committed to the Apache Storm open source community, as one of the co-authors of streamparse and pystorm.)