Re:Ordasity: Building Stateful Clustered Services on the JVM
by mitchellh on Thursday October 20, 2011 @01:54PM
(#3135966)
I'm not very familiar with building services on the JVM at the moment (though I'd love to be), so a quick, honest question:
What are the pros/cons of Ordasity[1] vs. something like Finagle[2]? Are they comparable and if so why would one be chosen over the other, what are the strengths of one vs the other, etc.
Re:Ordasity: Building Stateful Clustered Services on the JVM
by cscotta on Thursday October 20, 2011 @02:03PM
(#3136010)
Hey Mitchell,
Thanks for asking -- in short, the use cases are a bit different. Finagle is a framework for building asynchronous RPC systems (e.g., services or APIs with transports over HTTP or Thrift), and Ordasity is a library for cluster membership, load balancing, and distribution.
Twitter's put together a nice toolkit atop Netty for building reliable services and describing communication between them. These services are generally stateless and might be balanced by something like HAProxy (when using an HTTP transport), or via a round-robin approach in the case of Thrift clients. A closer comparator for Finagle would be something like Scalang, our library for building hybrid Erlang / Scala distributed systems (though it's also excellent for pure-Scala systems as well): https://github.com/boundary/scalang
Ordasity is designed for describing stateful clusters in which individual nodes are responsible for claiming longer-lived work units -- think of it in terms of "a processing shard" of the system's total load. The library's primary goals are to help you:
- Describe the cluster in simple terms such that when each node comes online, it joins the cluster and is aware of all other nodes (and vice versa)
- Distribute work across the cluster by directing each node to claim an even "count" of work units or "even distribution" of the total load imposed by those work units.
- Automatically rebalance work units across nodes in the cluster as the amount / intensity of work or cluster topology changes.
- And gracefully manage maintenance or downtime scenarios by draining work units to another node.
In the end, Ordasity and Finagle are both tools for building distributed systems on the JVM. However, the types of systems they're designed to describe are a bit different. Hope that helps; let me know if you'd like me to clarify something.
Re:Ordasity: Building Stateful Clustered Services on the JVM
by mey on Thursday October 20, 2011 @03:52PM
(#3136591)
Let me propose a use case, and I would appreciate if you indicated if I'm totally off base.
A virtual world, where actors(users) interact with each other, with the cluster providing validation and consistency between actors over a dynamic geography, where each node in the cluster would be handling the realtime requests of actors that are capable of interacting with each other, distant actors would be on different nodes.
(Would this work for a distributed server infrastructure to say an large online game?)
Re:Ordasity: Building Stateful Clustered Services on the JVM
by benblack on Thursday October 20, 2011 @02:24PM
(#3136130)
Much like our engineers, our software is extremely opinionated. Ordasity, Scalang, and Overlock are all small, extremely focused libraries to solve a single problem in a single way. This is quite similar to the Unix philosophy and we have found it to be similarly successful. Projects like Akka take a more maximalist, all-encompassing approach that doesn't work well for the way we build and operate services.
Re:Ordasity: Building Stateful Clustered Services on the JVM
by cscotta on Thursday October 20, 2011 @01:54PM
(#3135964)
Hi Roland,
Ordasity could certainly be used in building distributed JRuby applications. However, I'm not sure that pairing it with Rails would be the best use case.
Ordasity is designed for building stateful distributed services -- e.g., systems which can be described in terms of individual nodes serving a partition or shard of a cluster’s total load. Rails applications tend to follow a stateless model in which any application server could serve any request.
Ordasity is more appropriate for building services in which a specific node will be serving all [requests / queries / events] for a work unit in the cluster. In our case, that's our netflow aggregation systems (which pull data together from a network of edge nodes into a single Kafka stream), and our event stream processing system (which we shard by client). In this example, we have two Ordasity clusters -- one comprised of Kafka nodes, and a second containing our event stream processing tier. Ordasity handles work distribution across both tiers, ensures that both clusters are aware of each other and able to communicate, and helps us guarantee that everything is wired up properly (i.e., each node on each tier is communicating with the proper node on another tier) when data is passed between them.
In short -- yes, you could build a distributed service in JRuby (or Clojure, Mirah, Scala, Java, etc.) with Ordasity. Just wanted to offer that note to clarify use cases.
Re:Ordasity: Building Stateful Clustered Services on the JVM
by cscotta on Thursday October 20, 2011 @02:14PM
(#3136074)
If you're curious about Ordasity's load balancing strategies, check out Section 4: "Distribution / Coordination Strategy" in the docs: https://github.com/boundary/ordasity
The code is also fairly straightforward to follow if you'd like to take a glance in the repo as well.
Re:Ordasity: Building Stateful Clustered Services on the JVM
by Egregore on Thursday October 20, 2011 @05:27PM
(#3137138)
Thank you, I thought that you somehow found a way to measure CPU load of java processes, in fact you leave end user to measure the work load in end users units.
What are the pros/cons of Ordasity[1] vs. something like Finagle[2]? Are they comparable and if so why would one be chosen over the other, what are the strengths of one vs the other, etc.
[1]: https://github.com/boundary/ordasity
[2]: https://github.com/twitter/finagle
Thanks for asking -- in short, the use cases are a bit different. Finagle is a framework for building asynchronous RPC systems (e.g., services or APIs with transports over HTTP or Thrift), and Ordasity is a library for cluster membership, load balancing, and distribution.
Twitter's put together a nice toolkit atop Netty for building reliable services and describing communication between them. These services are generally stateless and might be balanced by something like HAProxy (when using an HTTP transport), or via a round-robin approach in the case of Thrift clients. A closer comparator for Finagle would be something like Scalang, our library for building hybrid Erlang / Scala distributed systems (though it's also excellent for pure-Scala systems as well): https://github.com/boundary/scalang
Ordasity is designed for describing stateful clusters in which individual nodes are responsible for claiming longer-lived work units -- think of it in terms of "a processing shard" of the system's total load. The library's primary goals are to help you:
- Describe the cluster in simple terms such that when each node comes online, it joins the cluster and is aware of all other nodes (and vice versa)
- Distribute work across the cluster by directing each node to claim an even "count" of work units or "even distribution" of the total load imposed by those work units.
- Automatically rebalance work units across nodes in the cluster as the amount / intensity of work or cluster topology changes.
- And gracefully manage maintenance or downtime scenarios by draining work units to another node.
In the end, Ordasity and Finagle are both tools for building distributed systems on the JVM. However, the types of systems they're designed to describe are a bit different. Hope that helps; let me know if you'd like me to clarify something.
A virtual world, where actors(users) interact with each other, with the cluster providing validation and consistency between actors over a dynamic geography, where each node in the cluster would be handling the realtime requests of actors that are capable of interacting with each other, distant actors would be on different nodes.
(Would this work for a distributed server infrastructure to say an large online game?)
One love,
Lil' B
It would be ideal to support Thrift protocol transports too
Ordasity could certainly be used in building distributed JRuby applications. However, I'm not sure that pairing it with Rails would be the best use case.
Ordasity is designed for building stateful distributed services -- e.g., systems which can be described in terms of individual nodes serving a partition or shard of a cluster’s total load. Rails applications tend to follow a stateless model in which any application server could serve any request.
Ordasity is more appropriate for building services in which a specific node will be serving all [requests / queries / events] for a work unit in the cluster. In our case, that's our netflow aggregation systems (which pull data together from a network of edge nodes into a single Kafka stream), and our event stream processing system (which we shard by client). In this example, we have two Ordasity clusters -- one comprised of Kafka nodes, and a second containing our event stream processing tier. Ordasity handles work distribution across both tiers, ensures that both clusters are aware of each other and able to communicate, and helps us guarantee that everything is wired up properly (i.e., each node on each tier is communicating with the proper node on another tier) when data is passed between them.
In short -- yes, you could build a distributed service in JRuby (or Clojure, Mirah, Scala, Java, etc.) with Ordasity. Just wanted to offer that note to clarify use cases.
The code is also fairly straightforward to follow if you'd like to take a glance in the repo as well.
Good work Scott and team!