News for hackers. Stuff that matters.
Re:Load Test Rails4 app ruby 2.1 MRI Vs Rubinius 2.0 by cpuguy83 on Wednesday October 09, 2013 @09:32AM (#6521089)
This is in no way a good test. Yes, Rubinius takes longer to load than MRI. It'a also JITed, so it needs time to warm up to start screaming.

Also, this is in dev mode... did you disable Rack::Lock?

MRI can only handle 1 request at a time, Rubinius can handle whatever your concurrency level is (assuming you've removed Rack::Lock from your environment)

Re:Load Test Rails4 app ruby 2.1 MRI Vs Rubinius 2.0 by timdorr on Wednesday October 09, 2013 @09:41AM (#6521147)
> This is in no way a good test. Yes, Rubinius takes longer to load than MRI. It'a also JITed, so it needs time to warm up to start screaming.

This is a fair point. You should run ab multiple times over a longer duration of time. 5 seconds seems like a short time to declare an absolute winner.

Re:Load Test Rails4 app ruby 2.1 MRI Vs Rubinius 2.0 by resca79 on Wednesday October 09, 2013 @09:47AM (#6521176)
Honestly I declare no winner. This is a very simple test. I wanted to share with you, to even get tips thanks
Re:Load Test Rails4 app ruby 2.1 MRI Vs Rubinius 2.0 by resca79 on Wednesday October 09, 2013 @09:37AM (#6521128)
Rails 4 is thread safe and I used puma as server Also the app is in production mode and the db pool is 16
Re:Load Test Rails4 app ruby 2.1 MRI Vs Rubinius 2.0 by timdorr on Wednesday October 09, 2013 @09:43AM (#6521158)
Can you post a link to the app? Have you considered running the TechEmpower benchmark app? https://github.com/TechEmpower/FrameworkBenchmarks
Re:Load Test Rails4 app ruby 2.1 MRI Vs Rubinius 2.0 by fbernier on Wednesday October 09, 2013 @09:44AM (#6521168)
Rails 4.0.0 application starting in development on http://0.0.0.0:3000

This most likely means Rack::Lock is used in your test.

Re:Load Test Rails4 app ruby 2.1 MRI Vs Rubinius 2.0 by resca79 on Wednesday October 09, 2013 @09:49AM (#6521187)
I used the following command rails s puma -e production
Re:Load Test Rails4 app ruby 2.1 MRI Vs Rubinius 2.0 by rubiquity on Wednesday October 09, 2013 @12:06PM (#6522111)
Rack::Lock was removed as a default in Rails 4.
Re:Load Test Rails4 app ruby 2.1 MRI Vs Rubinius 2.0 by rubiquity on Wednesday October 09, 2013 @11:57AM (#6522047)
It's a Rails 4 app. Rack::Lock is removed by default.
Re:Load Test Rails4 app ruby 2.1 MRI Vs Rubinius 2.0 by jacques_chester on Wednesday October 09, 2013 @09:43AM (#6521161)
Reliable benchmarking is hard. There are actual journals about measuring computer system performance.

Some questions:

* How is startup time controlled for?

* Why only one run?

* Why so few requests?

* Why no warmup period?

* Why this particular configuration?

* What is the OS, hardware configuration?

* Why is it running in a development environment instead of a test or prod environment?

Re:Load Test Rails4 app ruby 2.1 MRI Vs Rubinius 2.0 by ebonfortress on Wednesday October 09, 2013 @10:36AM (#6521469)
I would really like to see some articles about proper benchmarking procedures at 1000+ requests per second. Are there any?
Re:Load Test Rails4 app ruby 2.1 MRI Vs Rubinius 2.0 by jacques_chester on Wednesday October 09, 2013 @10:42AM (#6521503)
The ACM's SIGMETRICS[1] are the main group publishing this kind of research. Some of it gets very specialised -- just picking a title from the most recent Performance Evaluation Review[2], I get "YOUQMON: a system for on-line monitoring of YouTube QoE in operational 3G networks".

However, being the ACM, it's basically inaccessible unless you sell a kidney.

An organisation with a slightly more industrial focus is the Computer Measurement Group[3]. Like the ACM ... bring money.

[1] http://www.sigmetrics.org/

[2] http://dl.acm.org/citation.cfm?id=2518025&CFID=252561427&CFT...

[3] http://www.cmg.org/

Re:Load Test Rails4 app ruby 2.1 MRI Vs Rubinius 2.0 by look_lookatme on Wednesday October 09, 2013 @11:14AM (#6521720)
Sorry to go off-topic, but does anyone know if it's possible to get your hands on these pubs/journals for free at the Science, Industry and Business Library [1] in NYC?

[1] http://www.nypl.org/locations/sibl

Re:Load Test Rails4 app ruby 2.1 MRI Vs Rubinius 2.0 by resca79 on Wednesday October 09, 2013 @09:47AM (#6521179)
Prod. The app is a very simple rails4 standard app

iMac Intel Core 2 Duo 3,06 GHz cpu 1

Re:Load Test Rails4 app ruby 2.1 MRI Vs Rubinius 2.0 by jacques_chester on Wednesday October 09, 2013 @09:59AM (#6521237)

    => Rails 4.0.0 application starting in development on http://0.0.0.0:3000
Re:Load Test Rails4 app ruby 2.1 MRI Vs Rubinius 2.0 by resca79 on Wednesday October 09, 2013 @10:01AM (#6521246)
.... update the gits .. thanks!
Re:Load Test Rails4 app ruby 2.1 MRI Vs Rubinius 2.0 by warmwaffles on Wednesday October 09, 2013 @10:27AM (#6521407)
I would also love to see an application in production measured with NewRelic or something else and compare the results. A good way to test this would be to spin up three servers.

* 1 load balancer that plays round robin on the requests.

* 1 server running MRI

* 1 server running Rubinius

Each with NewRelic installed and compare the A/B test that is taking place for a period of about a week of heavy usage. Obviously the gems that the rubinius application would have would be different than what the MRI gems would be. But the core of the application would remain the same.

This would give me and others better insight as to how the two interpreters are running in a production.

Re:Load Test Rails4 app ruby 2.1 MRI Vs Rubinius 2.0 by bhauer on Wednesday October 09, 2013 @10:33AM (#6521446)
Although you may be running a single-threaded test intentionally, I generally recommend using a multi-threaded load simulation tool versus Apache's ab tool [1]. My preference is wrk [2], especially since it provides for time-limited tests rather than request-limited. Another alternative is WeigHTTP [3], which is basically a multi-threaded replacement for ab.

[1] https://news.ycombinator.com/item?id=6211883

[2] https://github.com/wg/wrk

[3] https://github.com/lighttpd/weighttp

Re:Load Test Rails4 app ruby 2.1 MRI Vs Rubinius 2.0 by JimmaDaRustla on Wednesday October 09, 2013 @09:18AM (#6521031)
So, MRI is way faster than Rubinius in this case? Interesting...
Re:Load Test Rails4 app ruby 2.1 MRI Vs Rubinius 2.0 by mrinterweb on Wednesday October 09, 2013 @10:18AM (#6521354)
Not necessarily. No warmup time was allowed for rubinius's cache. Also we don't know how concurrency was being handled. Rubinius should be able to utilize multiple cores. More information about the hardware, rails app, way that the puma server is being started, warmup time, duration of request time spent on the database, etc should be made available before any remote conclusion like which one is faster can be made.
Re:Load Test Rails4 app ruby 2.1 MRI Vs Rubinius 2.0 by JimmaDaRustla on Wednesday October 09, 2013 @10:36AM (#6521471)
That was my concern - Rubunius' selling point is concurrency, so I'm sure there is an approach to implementation, which is why I've never used it.
Re:Load Test Rails4 app ruby 2.1 MRI Vs Rubinius 2.0 by warmwaffles on Wednesday October 09, 2013 @10:24AM (#6521384)
No. Rubinius' JIT has not had time to warm up. These tests are a little unreliable. I can't find the article off hand, but the way these tests were done aren't exactly telling me anything. Power of ten tests don't do much other than show me that the tester is naive.
Re:Load Test Rails4 app ruby 2.1 MRI Vs Rubinius 2.0 by resca79 on Wednesday October 09, 2013 @09:20AM (#6521041)
yes... and I repeat it and sometimes in this case is 2x faster
Re:Load Test Rails4 app ruby 2.1 MRI Vs Rubinius 2.0 by ksec on Wednesday October 09, 2013 @09:34AM (#6521105)
Gist title is Ruby 2.0 MRI vs Rubinius 2.0 While HN Title is Ruby 2.1

So which is correct?

Re:Load Test Rails4 app ruby 2.1 MRI Vs Rubinius 2.0 by resca79 on Wednesday October 09, 2013 @09:35AM (#6521111)
sorry.. I have updated the test with ruby-head Ruby 2.1 MRI 2.1 Rubinius 2.0
Re:Load Test Rails4 app ruby 2.1 MRI Vs Rubinius 2.0 by harlock81 on Wednesday October 09, 2013 @09:13AM (#6521013)
very interesting, thanks!