I think there's an analogy between the two issues you brought up.

1. A concurrent garbage collector is 10x more work than a single-threaded one. People often don't realize this.

2. A language-independent VM is 10x more work than a VM for a given language. People often don't realize this this.

In other words, VMs are tightly coupled to the language they implement, unless you make heroic efforts to ensure otherwise.

WebAssembly is a good example of #2. I think the team is doing a great job, but they are inevitably caught between the constraints of different languages (GC, exceptions, etc.)

The recent submission The Early History of F# sheds some light on this with respect to the CLR:

https://news.ycombinator.com/item?id=18874796

An outreach project called “Project 7” was initiated: the aim was to bring seven commercial languages and seven academic languages to target Lightning at launch. While in some ways this was a marketing activity, there was also serious belief and intent. For help with defining the academic languages James Plamondon turned to Microsoft Research (MSR).

I think this is the only way to design a language-independent VM -- port a whole bunch of languages to it. And there are probably 4 or 5 companies in the world with the resources to do this.

I've seen some VM designs that aim to be generic, but since they were never tested, the authors are mistaken about the range of languages they could efficiently support.

Of course, you can always make a language run on a given VM, but making it run efficiently is the main problem.

> I'd love to see a better separation of language and VMs. I think it's a bit sad that a language designer has to either implement their runtime system from scratch, or has to run it on top of a VM that was designed for another language (Java in the case of Jython).

Wasn't Perl 6's Parrot kind of meant to fulfil that role?

> Wasn't Perl 6's Parrot kind of meant to fulfil that role?

Yes, that was an original project goal. You can see this as far back as Larry's State of the Onion 2003:

https://www.perl.com/pub/2003/07/16/soto2003.html/

... the "Parrot: Some Assembly Required" article written by Simon Cozens in September 2001:

https://www.perl.com/pub/2001/09/18/parrot.html/

... or, if you trust Git commits rather than articles which could have been edited in the meantime, the same article revised as introductory docs in the Parrot repository in December 2001:

https://github.com/parrot/parrot/blob/9bc8687beb5180e4cc8971...

I stand corrected.

I'd say not originally. But over time, as the Perl 6 project got delayed in the 2000's, it was decided that Parrot would be a runtime for all scripting languages. Which in turn meant it couldn't cater well enough for any. Which led to its demise.

Having recently been down this particular rabbit hole myself; I just want to note that there are other possible strategies, a GIL is not the only alternative to a fully concurrent runtime.

My own baby, Snigl [0]; doesn't even support preemptive threads, only cooperative multitasking; with a twist, since blocking operations are delegated to a background thread pool while yielding when multitasking.

[0] https://gitlab.com/sifoo/snigl#tasks

That is a nice strategy but it only allows IO to be parallel, and leaves the CPU sequential. Programs may be concurrent (because of the cooperative multitasking) but not parallel (because there is only one CPU thread). Users may find it disappointing that they can only use a single core.

Also, keep in mind that cooperative multitasking may cause unexpected high latencies, which is unfortunate e.g. in GUI applications and web servers; this is a result of queueing theory, and an example is given here: https://www.johndcook.com/blog/2008/10/21/what-happens-when-...

By the way, on POSIX systems there is a way to schedule background IO operations without even using threads: http://man7.org/linux/man-pages/man7/aio.7.html

I am aware, it's simply the most portable universal solution to the problem that I could think of. AIO comes with its own share of issues; Unix (and thus Posix) are pretty much dead; and it's not universal, I can't use it to open a database connection unless the database has support built-in.

From what I know, cooperative multitasking suffers significantly less from unpredictable performance than preemptive threading. The biggest source of uncertainty in Snigl is the preemptive IO loop.

The things is that I really don't feel like writing a concurrent runtime; been there, done that. I'm planning something along the lines of Erlang's processes and channels based on separate interpreters for those scenarios.

Ok. A problem with separate processes is that you have to serialize your data when passing messages. This is a pity because functional data structures + structural sharing can be very efficient. For example, I can't imagine how someone would implement a high performance database without structural sharing.

Processes by name, from Erlang.

They're implemented (in my mind so far) as preemptive threads, one per interpreter; which makes them slightly more heavy-weight than Erlang's NxM and a nice complement to purely cooperative multitasking.

Sounds interesting. I'm not familiar with Erlang, and I still wonder how shared memory is managed.

It's one of those languages that does things differently to solve actual issues, not to check boxes.

From my limited experience, Erlang doesn't share data between processes; you throw it over the fence by sending to the process inbox, which is where the locking takes place.

Still, shuffling data between OS threads is an easier problem to solve than serializing between OS processes.

> Therefore, the thing I'm looking forward to most is a concurrent, generic and portable VM written in Rust.

There is an effort to get the BEAM ported to rust, which would be very exciting.

Do you have a link to this info?

It might have been just one guy, but it was here on hn.

I've seen this one before here: https://github.com/kvakvs/ErlangRT

I am working on something similar but from a slightly different direction. The project is mainly focusing on compiler infrastructure for now, but I have a reference interpreter I use for validation. The short term goal is to make it able to run the erlang `compile` module. https://github.com/hansihe/core_erlang