How many languages are there like this? A handful? Erlang, Scala, Haskell, Go.... Java has always been parallel but with pretty primitive concurrency tools, no?
Java 5 got the concurrent package, with support for futures, chained computations, ability to create ou own schedulers, only matched by .NET dataflows.
Java 8, integrated the concurrent package with parallel streams.
Recent versions are being refactored so that eventually virtual threads are surfaced across all runtime, so instead of not knowing if green or red threads are being used like in the early Java days, the developers get to choose which ones to use.
On .NET side, naturally it copied the Java primitives, then came TPL, Dataflow, async/await, PLINQ, F# agents, F# async/await (not the same as standard async/await, based on F# computations).Additionally there is the research done with Axum, Cω, Singularity, Midori, Immutable C#.
For _concurrency_ (what the Ruby fiber schedulers are for) what is proposed is probably quite a good idea. We can do as much hatin' as we want on JavaScript but its concurrency story has been much more successful than Ruby's due to evented IO as well - even though everything is mutable. As far as Ractor is concerned - well, WebWorkers are kind of similar - they get a separate thread, but you can only send them "messages" which must be immutable. How the MRI team is going to tackle mutations in those messages is an interesting excercise, because Ruby is ultimately a _live runtime_ - i.e. most of the things can change at any time. Not because it is made for sloppy programming but because it is made for incremental changes to the VM like Smalltalk.
Object.freeze, presumably.
Immutability exists as an option within Ruby’s OOP system, and has since the beginning, so it’s well-integrated[1], with other language features having been built to take it into consideration.
I would expect any[2] message sent to a Ractor to be
1. On the sending side, verified as being acyclic;
2. On the receiving side, reconstructed by doing a recursive “clone and then freeze” operation.
Since most messages wouldn’t be “deep”, this wouldn’t usually be costly; but in the case of a “deep”(ly hierarchical) message, you could probably pre-“deep-clone-and-freeze” the objects you’re going to send, and the runtime would hopefully detect this and not bother to deep-clone-and-freeze them again. (Maybe they’d add a runtime tag-bit on objects for “me and all my references — transitively! — are frozen”, that could be pre-set for any object when it’s frozen, if it sees that all its instance-variables are also already marked as transitively-frozen.)
Alternately, they might just lean on the Marshal module, adding the capability for Marshal to load a dumped object-graph as all-frozen; and then just use Marshal.dump on the sending side and Marshal.load(..., freeze: true) on the receiving side. This wouldn’t allow for the efficiency wins of the above, but it would have the advantage of being a single already-well-tested-and-optimized all-in-C transformation; and it would give the MRI maintainers some docs to point at (those of Marshal.dump) to indicate what’s possible to send in a Ractor message.
—————
[1] Integers, Floats, and Symbols come frozen by default; and there’s a pragma you can add to source files such that their string-literals will also be created frozen. So it’s not like Ruby code isn’t “prepared” for frozen objects. Your own Ruby code deals with plenty of them!
[2] What if you want a mutable reference to an object in another thread? Well, think about the semantics of what you want. Presumably, you want sending a message to the object to actually first send (a recursive frozen clone) of your message back to the object’s owner-Ractor to handle. What system already does this? DRb! You can send a DRb object-proxy handle to another Ractor, and it won’t matter that it receives a recursively-frozen clone of it; it’ll still work to communicate with your own Ractor’s mutable “remote” object. Think of DRb handles as the Erlang PID-objects of Ruby.
While a lot of people scream that "you should not monkeypatch" (and it is pretty much always a good idea not to) it is not always practical, given the fact that a lot of software produced is imperfect and does need careful patching sometimes, and Ruby is great at allowing it.
But even having the basic threading primitives built in from a early point would help I think (e.g. Java, C++).
Why? Because it does not let library authors entertain the idea that they can just ignore all parallelism concerns.
> It will likely take some efforts and at least a year of work from the community to push libraries towards less shared state.
About one year to rewrite the ecosystem to fit this new threading model. Does that strike you as particularly realistic?
Like I said, I very much wish they would succeed.
Most objects are Unshareable objects, so you don't need to care about thread-safety problem which is caused by sharing.
Matz[1] released the first version of Ruby in Dec 1995.
DHH was a major player in getting Ruby into the global spotlight with Rails[2] in 2004. Rails got very popular as a framework for developing new applications, with Basecamp being novel, showing that it could work well and introducing people to REST, in a flexible interpretation, as well as ActiveRecord, whose ease of use and migrations became a model for modern web development.
Rails v3 divided the community, specifically around how and what Rails would support for the server and request-handling. This hinted at problems to come, but Rails was still strong, and many took it with a grain of salt and upgraded.
However, Twitter, which had been built on Rails became popular, and the "fail whale" emerged as they were unable to handle all of the requests. This was not a problem with scaling Rails, but with them knowing how they could scale Rails without much greater expense, but since they had to rewrite things and there was pressure to get scaling done right, they switched to Scala and Java, since Scala was functional and fast, and there was a lot of support for the JVM. Functional programming had already been making a comeback in popularity in the 2000s, because it often required a lower memory footprint and was fast. But, at that point in time, many teams and developers were looking into it.
Though it wasn't the first time he'd done optimization, in 2012, Matz released mruby[1][3], an embedded Ruby.
Around the same time, with functional programming having been cool, Elixir was born and some of the Rails community left for writing Ruby/Rails-ish code in Erlang.
Some had been trying to slim down Rails in core, so that there would be less code needed to serve requests.
Tenderlove, who came from the system programming side of things, joined the Rails core team with a focus on optimization, did work on Rack, and eventually he started working to help speed up Ruby.
For years, Matz and others had focused on speeding up and slimming down Ruby. Ruby had run on Lighttpd and Ruby on Rails could run on it also.
All of these things have been driving Ruby to get better, and now it is.
So, no, I don't think it's realistic that they put a year into it. At least 9+ calendar years led to this point, and it's been 26+ calendar years since initial release. And this isn't the end of it. It's not trying to compete with or tank your favorite framework or language of choice, it's just been improving and its team, even as good as it already was, has been improving.
P.S.- Ruby is not Rails. But not talking about how the history of Rails in the scope of things would be remiss. I can't think of anything in the history of Ruby that has been bad, but certainly Rails has had its "fun". But right now, it's coming together. I also didn't mention Sinatra's influence on slimming things down, or Puppet, Chef, etc.'s contribution to the Ruby community, or Crystal which has been a valiant effort for a compiled Ruby-like ecosystem. There is so much that happened leading to today that shaped where things are and where they are going. I'm totally psyched about this.
[1]- https://en.wikipedia.org/wiki/Yukihiro_Matsumoto
In some languages functionality is integrated and in some languages people developed it as a library. Which one is the right thing to do probably depends on what you can work with in the language.
Just to name some examples that come to mind: Erlang (messag passing to actors), Go (hyped go routines), Rust (multiple frameworks, Bastion comes to mind), Guile (guile-fibers, a library, because the language offers the primitives), Pony (actors, similar to Erlang), Elixir (on top of Erlang)
I do not know sufficient Ruby, to tell whether or not it offers the primitives required to implement this stuff as a library. If it does, then there is a higher chance of anyone eventually getting it right. If it does not, it will depend on the language designers to get it right.
Good concurrency support lets you handle thousands (even millions) of these activities in a single process, which is massively more efficient than using a process each, and puts less load on other systems (DB, Splunk, Graphite, RabbitMQ…)
Green threads in particular make it possible to get these benefits without writing the code in async style. You just write regular, blocking code and the runtime / frameworks take care of yielding to the scheduler (event loop) when appropriate.
So did it run Rails?
If you're talking about producing a static binary then both Rubymotion and TruffleRuby can do this.
Ruby has already been doing threaded dispatch on bytecodes since 1.9.
Call-threading looks like this:
while(1) { interpreter_state = *virtual_program_counter++(interpreter_state); }
There's an indirect jump here.With context-threading you JIT very simple code which looks like this and has only direct calls:
interpreter_state = put1(interpreter_state);
interpreter_state = put1(interpreter_state);
interpreter_state = plus(interpreter_state);You can definitely compile anything... whether it's worthwhile is a totally different story. I learned this the hard way :)
As far as I remember the Ruby 3x3 effort yielded 3x on SOME workloads, with tremendous effort, which is about what I would have expected due to Ruby's very dynamic semantics.
The CRuby JIT for 3x3 built by one person in their spare time.
https://developers.redhat.com/blog/2020/01/20/mir-a-lightwei...
I'm just not convinced that good parallellism is something you can successfully bolt on to a language 25 years down the road. The languages I've seen do a really good job all embraced parallelism as an important problem from the start and had a strong policy of immutability by default and good persistent data structures. Such a clear commitment permeates the ecosystem and you get libraries with clear parallelism stories .
Hopefully I'm wrong and this will all work out great for ruby.