> I wouldn't use Rust for the syntax; I'd use it because I want properties like complete memory safety, manual control over memory allocation, easy (and fast!) access to C libraries, and short startup time. These are all things that Truffle explicitly does not deliver.
Truffle itself does not do it, but it is possible to have a Truffle interpreter with these requirements. Memory safety can be enforced at the frontend compiler level like Rust does I believe. Memory can be allocated directly with Unsafe or calling libc functions. GNFI [1] allows for fast calls to native libraries. Startup can be improved radically with SubstrateVM, but it is currently closed-source. There are many ways to improve the JVM startup, but not necessarily convenient (but then you don't need to wait to compile your program either to start running it).
Pretty much every professional job I've had used polyglot programming of some sort - when I was in financial software it was mixed Java/C/Fortran numerics, when I founded my first startup it was polyglot Javascript/ActionScript, and at Google it was a large server that was first Python/C++ and then Java/C++. The boundary between languages is always problematic. And the reason for that is because you choose memory layout to make certain trade-offs around the access patterns for that data. Do you inline data structures in a vector, or chase pointers? Do objects carry their type information with them so you can manipulate them dynamically, or do they throw it away at compile-time for greater efficiency? Do you get O(1) string indexing or native UTF-8? Do you allocate on the stack or the heap? Do you copy, borrow, move, or COW?
And then when you go to switch representations to call into another language, you pay a cost to convert all the data structures you might be touching. In many cases, that cost could be more than you saved by using optimized representations in the first place.
An example is JRuby+Truffle where in its C extensions, pointers to Ruby objects act to the C code as if they are pointers to MRI data structures, but behind the scenes Graal compiles code that accesses them like the JRuby objects that they are.
The C-memory to Java GC transition doesn't work as well, but you can get around that by implementing C-memory using the Java GC like the Managed-C paper does, alternative implementations only have to keep the same behaviour, not the same underlying data representation.
What if it's not the C extension in your project that does the allocation, but a closed-source third-party library that your C code calls into? Do you need to recompile all source code to generate the appropriate typemaps?
If you already buy the JVM's premise of "just let the compiler do it, we'll figure out the most efficient representation", then this is a non-issue. But there are still programmers out there who believe that Rust or C++ or Go or CPython or whatever presents a better memory layout for the tasks that they wish to accomplish, or language designers who think they can do better than all of the above, and these are the users that Graal+Truffle is trying to win over. What's the story for them?
Sorta like SWIG++? If you could do SWIG but never require that an end-user write a typemap or debug a crash themself, there'd probably be a big market for that.
Not actually true. `ByteBuffer` lets you read integers from arbitrary offsets, which is all Cap'n Proto really wants.
"Little languages" can have significant advantages over APIs in terms of flexibility, optimization, and conceptual fit with specific domains. Why not create infrastructure to not only de-duplicate work creating tooling for languages while ensuring everyone always had 1st-class tooling and making "little languages" as easy to create as APIs?
It's perfectly possible and manageable to create cross-language VMs and to translate pretty accurately between languages. However, what breaks down, as you alluded to, is that people expect the standard libraries, and the libraries of others, to work interchangeably, and that is orders of magnitudes more complex (mostly because you are, by definition, working closer to the metal).
So, I need to ask, how much research is being done in giving the standard library as much consideration as the language? Do we have a grasp on what it would take to treat the stdlib as a "first class" citizen?
One of the solutions to this problem could be having package management in your language be a first-class citizen, then treating each part of the stdlib as a separate package. If using stdlib code is just as simple as using 3rd-party libraries, then why need a stdlib at all?
Without a decent stdlib, you end up with the pre-STL C++ situation, where every library & framework declares its own string, vector, and hashmap classes and you need slow, verbose, and error-prone routines to convert between them. The main benefit of having these in the stdlib isn't that you don't have to re-implement them, it's that every third-party lib knows exactly which class represents the concepts of text, sequences, and dictionaries.
The same goes for many other types, eg. Promises, Paths, URIs, Dates, Input/OutputStreams, Iterators, etc. Indeed, when one of the stdlib types is misdesigned (eg. java.util.Date), you get an utter mess as the community converges on an alternative (eg. JodaTime) and then the alternative API is folded back into the stdlib (Java 8).
Luckily, we're learning just which types need to be in the stdlib and which can be outsourced to a third-party package manager. In particular, a lot of serialization/parsing formats are better off in a package manager, as long as the language defines an annotation system that can be used to define which fields must be saved. And giant systems like webservers, webframeworks, or RPC formats are often better off as a third-party library.
Except that it's not really exposing much of the JVM from what the article mentions, it just uses the JVM as a base. That is, Truffle languages can talk to other truffle languages, but I'm not sure how easy it is for a truffle language to talk to a JVM language.
From everything I read, it seems like people thought Parrot was a good idea so decided to rewrite/reimagine it using the JVM instead of C as a base.
As for not moving in the right direction, it was probably far too ambitious to try to develop something like Parrot and something like Perl 6 at the same time, with dependencies on each other. Both projects would have been better served if they did their own things, and Parrot was just seen as "a good candidate for a second implementation" from the Perl 6 perspective, rather than the primary target. It's hard to target a VM that's not done, and it's hard as a VM to support a language that isn't finalized (and has crazy requirements).
On IBM i the language surface is RPG, Cobol, C, C++ and Java.
On other ones you could eventually consider the micro-coded CPUs as a kind of cross language VMs.
> The JVM works if your language is mostly like Java (Java, Clojure, Scala, Kotlin).
I'm really curious why you're characterizing Clojure as "mostly like Java"?
"Mostly" in this case means semantically, not syntactically. It's things like concurrency model; floating point behavior; memory layout; FFI; semantics of basic libraries; performance characteristics; and level of dynamism. You can write a Python frontend for both the JVM and CLR, and it will look like Python but act like Java and .NET, respectively, with no guarantee that native CPython libraries will work on it.
The problem is that this is where basically all the interesting language-design research is. I wouldn't use Rust for the syntax; I'd use it because I want properties like complete memory safety, manual control over memory allocation, easy (and fast!) access to C libraries, and short startup time. These are all things that Truffle explicitly does not deliver.
It's a great tool if you work in the JVM ecosystem and want to play around as a language designer. But most of the interesting languages lately have created their own ecosystems, and they succeed by solving problems so fundamental that people will put up with having to learn a new ecosystem to gain the benefits they offer.