News for hackers. Stuff that matters.
Re:Ruby 2.1: Out-of-Band GC by munificent on Thursday January 30, 2014 @03:38PM (#7153210)
Anecdote time!

I've been hacking on a little scripting language[1] lately. To see how its performance compares, I run a few benchmarks[2] against other similar (dynamically typed, bytecode compiled) languages: Lua, LuaJIT (interpreted), Python, and Ruby.

Like others, I had internalized "Ruby is slow" through osmosis. But the version of Ruby I happen to have on my machine is 2.0.0.

In my little benchmarks, it turns out Ruby is one of the fastest. I haven't compared to 1.8.7, but I'm guessing that was much slower.

[1] https://github.com/munificent/wren [2] https://github.com/munificent/wren/tree/master/benchmark

If you want the gory details, here's the results of running them against Lua 5.3.2, LuaJIT 2.0.2, Python 2.7.5, and Ruby. (Yes, I should try against Python 3. I will.):

                          score time   wren score relative
    binary_trees - wren   3193  0.31s
    binary_trees - lua    1366  0.73s  233.72%
    binary_trees - luajit 6256  0.16s   51.04%
    binary_trees - python 1376  0.73s  231.96%
    binary_trees - ruby   3003  0.33s  106.34%

    fib - wren            3078  0.32s
    fib - lua             2785  0.36s  110.52%
    fib - luajit          6900  0.14s   44.62%
    fib - python          1331  0.75s  231.37%
    fib - ruby            3548  0.28s   86.77%

    for - wren            6080  0.16s
    for - lua             1990  0.08s   50.71%
    for - luajit          5914  0.02s   13.24%
    for - python          2825  0.35s  215.25%
    for - ruby            6595  0.15s   92.19%

    method_call - wren    4707  0.21s
    method_call - lua     1674  0.60s  281.17%
    method_call - luajit  4221  0.24s  111.53%
    method_call - python   767  1.30s  613.69%
    method_call - ruby    3061  0.33s  153.77%
As you can see, Ruby fares quite well.
Re:Ruby 2.1: Out-of-Band GC by alberth on Thursday January 30, 2014 @04:41PM (#7153525)
>> "To see how its performance compares, I run a few benchmarks against ... LuaJIT (interpreted)"

Why would you use LuaJIT in interpreted mode?

That defeats the main reason for using LuaJIT, and you'll see massive improvement in speed if you run it not interpreted [1]

It should also be noted that, even in the slower interpreted mode of LuaJIT - it was still the fastest in completing your benchmarks compared to all other languages/implementations. It will run even faster if you don't run LuaJIT in interpreted mode.

[1] http://luajit.org/performance_x86.html

Re:Ruby 2.1: Out-of-Band GC by munificent on Thursday January 30, 2014 @06:24PM (#7154099)
Running LuaJIT interpreted is a deliberate choice.

Some platforms, in particular iOS and game consoles, do not allow generated machine code. I want my language to be usable for games, and that's one reason I'm using bytecode. (Simplicity is the main reason.)

Given that, I think it's most helpful for me to compare Wren's performance to other bytecode language implementations. Comparing to JIT compiled languages is a bit apples/oranges. They tend to be something like an order of magnitude faster (and an order of magnitude more complex!).

Meanwhile, LuaJIT's interpreted mode is very fast as you note. I think it's one of the fastest bytecode interpreters around, so it's a great target for Wren to aim for.

If you want a lot more detail on my thoughts about this, see here:

https://github.com/munificent/wren/blob/master/doc/site/perf...

Re:Ruby 2.1: Out-of-Band GC by the8472 on Sunday February 02, 2014 @05:48AM (#7165609)
What makes ruby (MRI particularly) slow are metaprogramming things that slow it down when done at runtime (dynamically definding methods and classes), its global interpreter lock (preventing proper parallelism) and the single-threaded super-slow GC.

Integer math is fast because they're using tagged pointers and thus don't need to allocate real objects for them. Method calls also are reasonably optimized, although they don't have inline caches afaik.

You're basically testing a few tiny bits of the runtime that happen to be reasonably fast. The big frameworks like rails do a lot of stuff that is horribly inefficient. Combined the aforementioned single-threadedness and slow GC leads to ruby being slow in real-world applications even if a few microbenchmarks are ok for an interpreted language.

Re:Ruby 2.1: Out-of-Band GC by mcguire on Thursday January 30, 2014 @04:03PM (#7153340)
"dynamically typed, bytecode compiled"

I haven't followed Ruby development, but the last time I heard, Ruby 1.8 and prior was a tree-walking interpreter---simple to implement but not fast at all.

Re:Ruby 2.1: Out-of-Band GC by srd on Thursday January 30, 2014 @04:41PM (#7153526)
This changed in 1.9. Ruby is now compiled into a stack machine based bytecode. Ruby 1.8 used the AST internally, you're right there. This made e.g. method calls very slow compared to the new byte code representation.
Re:Ruby 2.1: Out-of-Band GC by steveklabnik on Thursday January 30, 2014 @04:50PM (#7153557)
1.8 -> 1.9 was an entire re-write, it's now a bytecode compiled language: http://www.ruby-doc.org/core-2.1.0/RubyVM/InstructionSequenc...
Re:Ruby 2.1: Out-of-Band GC by munificent on Thursday January 30, 2014 @04:41PM (#7153522)
That's correct. I still think Ruby 1.8 would be a useful data point just because it's a widely used language that many people are familiar with, but in this case I'm comparing it to Ruby 2.0, so it is apples/apples.

For what it's worth, I look at these benchmarks as validating feasibility more so than really trying to win some performance fight between languages. In order to argue that my language is suitable for real world use, I need to show it's performance is comparable to other languages that are widely used. From that angle, Ruby is a fine comparison, regardless of how it's implemented.