Between Ruby 1.8 and 2.5, performance has improved around 13x in tight loops[2]. The Rails performance issue has been massively overblown since 1.9 was released.
Ruby 1.8 was a tree walking interpreter, so the move to a bytecode VM in 1.9 was a huge leap in performance. Twitter bailed to the JVM before moving to 1.9. A lot of those 10-100x performance differences to the JVM are gone thanks to the bytecode VM and generational GC.
Bytecode VMs all have the same fundamental problem of instruction dispatch overhead, they're basically executing different C functions depending on input.
Doing _anything_ to reduce this improves performance dramatically, even just spitting out the instruction source code into a giant C function, compiling it, and calling that in place of the original method. Another 10x improvement on tight loops should not be a problem.
[1] https://www.techempower.com/benchmarks/#section=data-r15&hw=...
[2] https://github.com/mame/optcarrot/blob/master/doc/benchmark....
Nor did i know that twitter jumped out of rails before ruby got performant. Which means the argument that twitter outgrew rails isn't so correct anymore.
still, thanks for this insightful comment.
Twitter, even back in those days would have still outgrown today's Rails. It was Ruby that has gotten a lot faster. Not necessarily Rails.
Yeah no kidding.. https://samsaffron.com/archive/2018/06/01/an-analysis-of-mem...
Ideally, you'd serialize directly from the database, bypassing the application entirely. Easily doable in ActiveRecord, but it's an explicit action, not the default. Not even sure if it's available in other databases besides PostgreSQL.
With 2.6 and sorbet [1] coming down the line, it's exciting to be a Rubyist again!
It does if you ignore the overhead of JIT compilation itself. However, my understanding is that writing a JIT implementation that performs better than a good interpreter is surprisingly difficult. You have to have a lot of complicated logic for tracking hotspots and using JIT judiciously in short-running scripts.
It's the instruction dispatch overhead that's the real unavoidable problem. LuaJIT, for example, uses a bunch of tricks to minimize it in the bytecode VM, and it's significantly faster than the standard Lua VM but still far, far slower than basic JIT compilation.
Lua JIT is one of the most sophisticated dynamic language JITs out there, so it's hardly evidence that a simple implementation of a JIT will perform better than a good bytecode interpreter.
The problem is less acute for server side apps because the programs run for a long time, so that the initial compilation overhead is insignificant. However, there's a reason that you need a JIT to make Ruby fast rather than an ahead of time compiler. Ruby has so few compile-time guarantees that you need to do a lot of dynamic specialization to get really significant performance improvements. So compilation might still be triggered even after a script has been running for a long time.
I'd add that PyPy, which is also very sophisticated, is often not much faster than CPython, and in fact is slower for some types of code. Writing good JIT-based implementations for dynamic languages is really a tough problem. See e.g. the following post for some explanation of why:
Yes.
> Lua JIT is one of the most sophisticated dynamic language JITs out there, so it's hardly evidence that a simple implementation of a JIT will perform better than a good bytecode interpreter.
I meant that even a basic JIT can offer the same speedup as LuaJIT's interpreter, and a lot more work went into the latter.
> The problem is less acute for server side apps because the programs run for a long time, so that the initial compilation overhead is insignificant. However, there's a reason that you need a JIT to make Ruby fast rather than an ahead of time compiler. Ruby has so few compile-time guarantees that you need to do a lot of dynamic specialization to get really significant performance improvements. So compilation might still be triggered even after a script has been running for a long time.
The initial results of MJIT for simply removing the instruction dispatch overhead and doing some basic optimizations are a 30-230% performance increase on a small but real-world benchmark. No type specialization and specular optimization required.
> I'd add that PyPy, which is also very sophisticated, is often not much faster than CPython, and in fact is slower for some types of code. Writing good JIT-based implementations for dynamic languages is really a tough problem. See e.g. the following post for some explanation of why:
Most of the discussion about PyPy is completely irrelevant for the discussion about MJIT. PyPy isn't a method JIT. PyPy traces the interpreter itself and tries to produce a specialized interpreter. It works even worse at optimizing Ruby code via Topaz.
Topaz was easily the fastest Ruby JIT before TruffleRuby, beating the JRuby and Rubinius JITs. It was very impressive.
It'a shame Topaz was never really "finished".
So, this amounts to a small improvement for some types of code. Indeed, it is "easy" to get that by "just" using some basic JIT techniques. The trick is to get consistently better performance across the board. Relevant tweet at https://medium.com/@k0kubun/the-method-jit-compiler-for-ruby...:
>I've just committed the initial JIT compiler for Ruby. It's not still so fast yet (especially it's performing badly with Rails for now), but we have much time to improve it until Ruby 2.6 (or 3.0) release.
This will come with the rest of the opimizations Takashi has planned for Ruby 2.6. Ruby-Ruby method inlining, which is almost finished, is a huge one for improving Rails performance. IMHO there's no real point talking about Rails until it's working in some form.
> >I've just committed the initial JIT compiler for Ruby. It's not still so fast yet (especially it's performing badly with Rails for now), but we have much time to improve it until Ruby 2.6 (or 3.0) release.
It turned out this wasn't even testing MJIT with Rails because https://twitter.com/samsaffron/status/963219086833434624
Code doesn't even need to be "hot" to make it worth it. WebKit switches from interpreter to cheap baseline compilation, without an specular optimizations or type information, after only 6 calls of a function: https://webkit.org/blog/3362/introducing-the-webkit-ftl-jit/
This article literally describes how baseline JIT is worth it simply to remove the bytecode VM dispatch overhead.
Relevant post here:
https://rfk.id.au/blog/entry/pypy-js-faster-than-cpython/
A simple JIT can get you to the point where you reliably outperform a bytecode interpreter for certain types of code. What takes a lot more engineering effort is reliably performing at least as fast as a bytecode VM for all types of code.
PyPy is a much more ambitious design, completely replacing CPython, and using an unusual JIT scheme of tracing the interpreter itself and trying to produce an interpreter optimized for particular traces of your code.
It was much harder for that approach to reach the same level of general performance than it seems to have been for CRuby & MJIT.
Charles Nutter's early tests using JRuby on the GraalVM sound like there's another big step in performance coming without a huge amount of work.
https://www.techempower.com/benchmarks/#section=test&runid=a...
CRuby's GIL doesn't really matter for serving web requests since it's run with one process per core like NodeJS is. It's less memory efficient but doesn't really affect throughput so much. Also, JRuby has no GIL.
Another big win is the bootsnap gem, which is a cache of previous VM runs that loads faster than parsing all invariant pieces of code again.
I haven't had a chance to use Bootsnap yet but it sounds really promising.
Golang plus gin, sure. However there are other Go frameworks on the charts that blast the Ruby competition out of the water. Ruby isn't really on the podium at all with C, C++, Rust, Golang, C#, and Java about an order of magnitude out in the lead on fortunes.
Martini isn't much of a framework itself either, so lets forget the full featured nonsense. Almost none of the ecosystem is in play with these benchmarks. You could build a system up around fasthttp just as well as net/http, and ASP.NET certainly can't be accused of being a for-purpose contender.
The most impressive thing IMHO is how well Ruby is doing on maximum latency. I can't quite reconcile that considering fasthttp is pretty much zero-allocation and golangs stop the world is in the microseconds.. Pretty impressive.
Fast GC is critical to Ruby performance so a ton work went into it. Ruby 2.2+ has a very short STW phase thanks to generational GC + incremental marking.
Ruby (MRI) will have to reinvent the wheel in order to get a panoply of optimizations that some very smart people have already baked in: like the ability to target almost any platform from the same library.. GCC requires cross-compiling per target.
Test suite still passes on it though, so upgrading shouldn't be a huge deal at least へ‿(ツ)‿ㄏ
An order of magnitude as in .. 10x? This seems too good to be true. Half the arguments against Rails melt away like butter if that's truly the case.
Anyone with a better understanding of the details care to comment on the likelihood of these performance gains being actually realised, and if not, what we might realistically expect?