Onyx: fault tolerant data processing for Clojure

Re:Onyx: fault tolerant data processing for Clojure by pixelmonkey on Friday March 06, 2015 @12:52PM (#9158385)

Michael, can you explain this more? "[Storm] Topologies are written with functions, macros, and objects. These things are specific to a programming language, and make it hard to work at a distance -- specifically in the browser. JavaScript is the ultimate place to be when creating specifications."

I don't really get it. Storm Topologies are built in Java or Clojure using a builder interface, but the data structures for topologies themselves are actually DAGs that serialize using Thrift. It's true that this is a bit heavy-weight compared to something like JSON or EDN, but offering an alternative is a discussion in the community right now. What would your ideal representation of topologies be, actually?

[ Parent | on HN ]

Re:Onyx: fault tolerant data processing for Clojure by XPherior on Friday March 06, 2015 @01:00PM (#9158427)

I wasn't aware that they're Thrift serializable - that's cool, and offers roughly what Onyx does in terms of its workflow representation.

Onyx goes a little further though in terms of its catalog. I wanted more of the computation to be pulled out into a data structure. That includes runtime parameters, flow, performance tuning knobs, and grouping functions. All of these things are represented as data in Onyx. It's a little harder, at least in my experience, to do these things in Storm.

[ Parent | on HN ]