Re:Concurrency in Ruby almost as good as Node.js
by desireco42 on Wednesday August 17, 2011 @08:03PM
(#2897446)
I just can't resist, Serbs have saying 'almost is what makes girl unhappy' :).
On more serious note, ruby is definitely better when it comes to richness of available libraries which is what node is missing. I think in real life node can handle more, I heard that issues with memory leaking are solved, probably uses less memory as well.
I agree with conclusion that both are more then capable or handling async problems, I would use event machine on ruby end, this whole example seems flawed a bit. Anyhow, thanks for posting your results.
Re:Concurrency in Ruby almost as good as Node.js
by mrinterweb on Wednesday August 17, 2011 @05:44PM
(#2897092)
I would be interested to see how the unicorn variant, rainbows, would benchmark in this test. Rainbows seems much better suited for high concurrency applications because it utilizes fibers for concurrency as well as worker processes. http://rainbows.rubyforge.org/ Also after looking at the sinatra app's code, I believe all requests to MongoDB are going to be blocking. For the comparison to be more fair, the ruby app should be written to take advantage of Goliath, em-synchrony, event machine, or another non-blocking ruby fiber based option. Until the ruby app is non-blocking, this comparison is apples to oranges.
Re:Concurrency in Ruby almost as good as Node.js
by maxdemarzi on Wednesday August 17, 2011 @05:23PM
(#2897004)
This is great work but...please redo the charts to start at Zero and end at 50,000 to keep things consistent and prevent Edward Tufte from killing a kitten if he ever sees your charts.
Re:Concurrency in Ruby almost as good as Node.js
by cromulent on Thursday August 18, 2011 @12:16AM
(#2897928)
I thought Tufte was OK with non-zero baselines:
"The urge to contextualize the data is a good one, but context does not come from empty vertical space reaching down to zero, a number which does not even occur in a good many data sets. Instead, for context, show more data horizontally! "
Re:Concurrency in Ruby almost as good as Node.js
by maxdemarzi on Thursday August 18, 2011 @10:42AM
(#2899334)
A non-zero baseline is fine in some cases (line chart of the temperature of a person with a fever for example), but the issues here were multiple charts at varying Y scale starting points, and bar size comparisons of a property that we do want to go to zero (zero failures).
Re:Concurrency in Ruby almost as good as Node.js
by eric-hu on Wednesday August 17, 2011 @05:59PM
(#2897127)
Yeah, the graphs are misleading because of the varying Y scale.
I do like the conclusions section, though. It presented some insightful analysis of dyno/worker efficiency
>It could be that I’m benchmarking from only 1 server but I’ve seen almost no difference between having 40 dynos or 60. You’ll see one when you receive the bill so be cautious, especially if you use an auto-scale tool.
>The same applies to node with cluster, you can do more with 15 dynos running cluster with 3 workers than with 60 dynos of node alone (for a quarter of the price!).
Re:Concurrency in Ruby almost as good as Node.js
by ianloic on Wednesday August 17, 2011 @09:42PM
(#2897645)
But node.js doesn't have any concurrency. It's single-process, single-threaded and event driven, so by definition it's not concurrent. It's just not blocking.
Also, was someone really trying to run performance benchmarks on a shared machine running on a VM and expecting to get anything meaningful out? Really? Like really? I'm embarrassed for you.
Re:Concurrency in Ruby almost as good as Node.js
by kevingadd on Wednesday August 17, 2011 @05:16PM
(#2896978)
Hey, look, requests/s on a synthetic benchmark.
Let's not look at things like thread safety, resource consumption, average/best case/worst case latency, quality of available libraries and their support for concurrency, or code quality/maintainability issues.
Meh.
At least he provided the source code and plenty of data. That's cool.
Re:Concurrency in Ruby almost as good as Node.js
by moe on Wednesday August 17, 2011 @06:43PM
(#2897260)
I don't think we are required to provide a better analysis in order to point out the problems with a flawed one.
Whatever the author is measuring here, it's most certainly not "node" vs "ruby". This should be obvious when you look at the test-apps that he's using (which involve mongodb and serialization).
Re:Concurrency in Ruby almost as good as Node.js
by jshen on Wednesday August 17, 2011 @09:42PM
(#2897644)
It's very easy to complain, cast stones, etc. I don't think I've seen a benchmark on HN or reddit where many of the comments were not smug complaints. Yet, I've never seen any of the complainers produce a benchmark with the exception of zedshaw and one other person whose name I've forgotten.
Re:Concurrency in Ruby almost as good as Node.js
by RuadhanMc on Thursday August 18, 2011 @02:37AM
(#2898247)
Why should they?
Just because someone posts a benchmark that is obviously flawed, doesn't mean that someone else should be obligated to do their own benchmark as a counter.
These guys didn't ask for node.js to be used in any benchmark.
The onus is on the person who chooses to do a benchmark, and publish the results, to get it right. If they don't get it right, then they are in the wrong and deserve the criticism that they get as a result.
Re:Concurrency in Ruby almost as good as Node.js
by FooBarWidget on Thursday August 18, 2011 @04:06AM
(#2898394)
Because it's like complaining about the weather. It doesn't do anything to make things better. I think there's too much negativity on the Internet already.
Re:Concurrency in Ruby almost as good as Node.js
by RuadhanMc on Thursday August 18, 2011 @05:34AM
(#2898593)
There is too much negativity on the Internet, but there's also too much of just about everything else. The ease of publishing on the Internet means that people don't think too critically about their work prior to publishing it.
For example, if these guys were publishing this benchmark as part of a study or argument in a quarterly magazine -- and this was there only opportunity to publish it for the next year or so -- then they would have been far more critical of their own work prior to publishing it. Instead they've quickly thrown together something and pushed it onto the Internet without too much further thought.
Re:Concurrency in Ruby almost as good as Node.js
by pyre on Thursday August 18, 2011 @04:25AM
(#2898417)
To some extent this is like a peer-review. If I submit a paper to a journal, should my response to all criticism be, "Well, if you think I didn't do the experiment right, then you do it!"
Re:Concurrency in Ruby almost as good as Node.js
by rockarage on Wednesday August 17, 2011 @07:00PM
(#2897298)
This is not a Ruby vs Node.js benchmark because Node.js is still a layer above the V8 Javascript engine. Moreover Node.js was optimize for things like concurrency and this is similar to what eventmachine is optimized for. The difference is node.js makes the event model a language construct. I think it would have been better to use eventmachine but you did not because:
> At first I tried to compare bare node.js against eventmachine_httpserver. It did quickly became obvious that this kind of micro-benchmark wasn’t going to be very helpful in deciding which one to choose.
Your first observation was the correct one. The best thing to do is just pick a language and go with the best tools available.
Re:Concurrency in Ruby almost as good as Node.js
by chaostheory on Wednesday August 17, 2011 @05:57PM
(#2897122)
Yes the nodejs concurrency model takes some time getting used to, but working with threads is much harder imo. Raw speed benchmarks don't do enough to give a full picture.
Re:Concurrency in Ruby almost as good as Node.js
by mhd on Wednesday August 17, 2011 @06:17PM
(#2897192)
This is comparing nodejs with Ruby's EventMachine, so where do threads enter the picture? It's been a while since I looked at EM, but if I remember correctly, it's pretty much in the same area as Python's Twisted or node.js, as opposed to something like plain ol' Java threads or Erlang.
Re:Concurrency in Ruby almost as good as Node.js
by stock_toaster on Wednesday August 17, 2011 @11:52PM
(#2897893)
I don't actually even see eventmachine in there, aside from a remark about having tried it in the opening introduction. Seems it is just comparing node with sinatra+mongomapper running under thin or unicorn.
Based on the title I assumed it was something like goliath or sinatra/async with rainbows!/thin. Looking at the git repo linked in the article though, I don't believe that is the case.
Re:Concurrency in Ruby almost as good as Node.js
by simpsond on Wednesday August 17, 2011 @06:15PM
(#2897186)
EventMachine uses an event loop like node, so threads are somewhat irrelevant here. Unless "workers" are threads and not actual processes, but I am pretty sure they are processes.
On more serious note, ruby is definitely better when it comes to richness of available libraries which is what node is missing. I think in real life node can handle more, I heard that issues with memory leaking are solved, probably uses less memory as well. I agree with conclusion that both are more then capable or handling async problems, I would use event machine on ruby end, this whole example seems flawed a bit. Anyhow, thanks for posting your results.
"The urge to contextualize the data is a good one, but context does not come from empty vertical space reaching down to zero, a number which does not even occur in a good many data sets. Instead, for context, show more data horizontally! "
-- Edward Tufte, October 18, 2001
http://www.edwardtufte.com/bboard/q-and-a-fetch-msg?msg_id=0...
I do like the conclusions section, though. It presented some insightful analysis of dyno/worker efficiency
>It could be that I’m benchmarking from only 1 server but I’ve seen almost no difference between having 40 dynos or 60. You’ll see one when you receive the bill so be cautious, especially if you use an auto-scale tool.
>The same applies to node with cluster, you can do more with 15 dynos running cluster with 3 workers than with 60 dynos of node alone (for a quarter of the price!).
Also, was someone really trying to run performance benchmarks on a shared machine running on a VM and expecting to get anything meaningful out? Really? Like really? I'm embarrassed for you.
Let's not look at things like thread safety, resource consumption, average/best case/worst case latency, quality of available libraries and their support for concurrency, or code quality/maintainability issues.
Meh.
At least he provided the source code and plenty of data. That's cool.
Whatever the author is measuring here, it's most certainly not "node" vs "ruby". This should be obvious when you look at the test-apps that he's using (which involve mongodb and serialization).
Just because someone posts a benchmark that is obviously flawed, doesn't mean that someone else should be obligated to do their own benchmark as a counter.
These guys didn't ask for node.js to be used in any benchmark.
The onus is on the person who chooses to do a benchmark, and publish the results, to get it right. If they don't get it right, then they are in the wrong and deserve the criticism that they get as a result.
For example, if these guys were publishing this benchmark as part of a study or argument in a quarterly magazine -- and this was there only opportunity to publish it for the next year or so -- then they would have been far more critical of their own work prior to publishing it. Instead they've quickly thrown together something and pushed it onto the Internet without too much further thought.
So long as it is civil, criticism is fine.
> At first I tried to compare bare node.js against eventmachine_httpserver. It did quickly became obvious that this kind of micro-benchmark wasn’t going to be very helpful in deciding which one to choose.
Your first observation was the correct one. The best thing to do is just pick a language and go with the best tools available.
Based on the title I assumed it was something like goliath or sinatra/async with rainbows!/thin. Looking at the git repo linked in the article though, I don't believe that is the case.
number_of_threads * thread_stack_size = sadface.jpg (when trying to use threads to handle lots of connections)
For example, if he tested a 1 meg stream of chunked data being served on request, the node.js numbers would be very different.