Huh? The first part of the sentence makes me think of Rust. It's fast, memory safe, type safe, and thread safe.
Is GC a problem in language analysis systems? Doesn't feel that way these days, and anyway JVMs have pauseless GCs anyway now.
In terms of speed, as long as you don't choose Python I think you'll be fine.
I do think in terms of productivity, something like F#, that has GC, ML style syntax with piping and partial application, and a strict enough type system with Hindley-Milner type inference, would be the most productive.
My hold up was mostly with the sentence I quoted which seems to contradict itself.
Microsoft didn't go that way and again it feels like maybe JS limitations are the cause. Calling into a native library is tough in JS because there's no standardized FFI (browsers wouldn't allow it) and you're going to end up blocking the one JS thread you've got. Native code is going to expect the host to be able to spin up threads and use them because that's a super reasonable assumption, but one V8 can't meet. So they want everything to look like a web server because that's what JavaScript is intended for. That pushes them down a particular path, but it's definitely worth stepping back and saying ... does this really make the most sense, overall?
That's basically what separate processes are, so what's your point?
> Calling into a native library is tough in JS because there's no standardized FFI
No need for using anything standardized: VS Code brings its own custom browser engine – for better or for worse – so it's not limited to what any Web standards allow or forbid.
> you're going to end up blocking the one JS thread you've got
There are worker threads in JS.
JS has workers but they aren't sharing memory, so they aren't really threads.
The "shared state" in this context is IDE program text, and analyzer diagnostics. Copying those across address spaces and keeping them in sync is super fast, the overhead is too low to matter.
Recall that most analysis is done async, so results arrive at different times. Even if you're not doing anything then, your IDE/backend will be context switching back and forth as stuff arrives to be rendered.
In the limit this degrades to the backend being the entire IDE and the frontend being just an translator between the OS IO system a stream of JSON rendering commands.
The way I see it, there are possible two grand architectures:
* you either have a single, multi-language combine supporting any language. In a sense, LLVM/GCC but for front-ends
* or you have a bunch of independent servers
The first is clearly less code — there’s a lot to share. The second is embarrassingly parallel though. So, if you are a (single-threaded company), you should go for the first approach. If you are a (multi-threaded) OSS community, the second approach has shorter critical path.
Some second order effects:
1. While backands are more or less the same between languages and LLVM is a clear win, frontends tend to differ a lot. That’s the point of having different programming languages. So in practice sharing the code in IDE is harder. 2. An interesting benefit of separate process architecture is that it forces you to have async interface, which is important for responsive IDEs. But that’s an argument for separating back and front, not necessary for sharding back.
This would mean that you'd have to write a compiler not just for each host language/source language combination, but for every IDE/source language combination, because Atom would have different internal data structures than VS Code.
Historically for IntelliJ that meant writing everything in Java. Nowadays plugins are written in many different languages. You could write plugins in Ruby or Python if you wanted to. They run on the JVM. You could even use C++ and get the same benefits because GraalVM can run C++ on the JVM itself, i.e. it will apply GC to the C++ parts so you don't have to worry about ownership across the interface. But that'd be an advanced move and I don't think anyone has tried that; for such big gaps in language they tend to fall back on the more conservative multi-process model again. Stuff like Sulong is too new to have made such an impact.
Atom would have different internal data structures than VS Code.
LSP doesn't solve that. It either forces everyone to implement the LSP model and data structures, or you have a mismatch and translation anyway.
Why is that?
1. An in-process type safe API can be evolved and iterated rapidly. There's no need to write informal English specs for everything because the API itself provides so much guidance. Actually, IntelliJ is rather notorious for its absence of JavaDocs though this reflects cultural decisions inside JetBrains itself more than anything intrinsic (they take "code should be self documenting" perhaps a bit too seriously). In contrast, the LSP spec is enormous and the document itself doesn't provide much assistance implementing it (the article discusses this somewhat).
2. A protocol spec doesn't give you any re-usable code. If you write an IntelliJ plugin there's tons of re-usable helper logic and you can easily customize other plugins. The whole thing is designed for on-the-fly presentation compilers. With LSP you may or may not have anything to base your work on.
3. An IDE is fundamentally a highly parallel mutable database, in which the data can be in arbitrarily invalid forms at any point and the user is allowed to change anything whilst many different analyses are running in parallel. Shared data structures are everything in this problem, yet the LSP model effectively forbids all shared state between backend and frontend. This feels like a constraint inherited from browsers - it arguable doesn't make things easier, it just replaces relatively simple and statically/dynamically analyzable locking protocols with complicated sync and rec protocols. Both sides have to maintain their own copies of each other's data structures instead of re-using a single source of truth. This can get really messy, really fast. You're also constantly paying the costs and overheads of multi-process like context switching (especially expensive on Windows), IPC copies, process overhead, serialization etc.
4. A presentation compiler doesn't seem to benefit much from being written in Rust or C++. Developer workstations are fast and have plenty of RAM, so GCd/JITd languages don't seem like a particular problem here. Developer productivity on the other hand is at a premium. People want features. One reason JetBrains can add new languages and features so fast is that they're writing in relatively forgiving languages like Java and Kotlin, where you don't have to think too hard about ownership or memory management. Some thought is still needed - see the disposable model inside their IDEs - but you can more or less just go for it with a high rate of feature throughput.
You raise the question of how to kill hung operations. In practice that doesn't seem to be a common problem. I've been using IntelliJ with various languages for years and there have been many bugs, but I can't remember any time that a plugin hard-hung whilst analyzing. There are cases where plugin bugs happen but exceptions and GC mean the IDE can simply catch the errors and disable it. These days it can do that temporarily and simply reload the plugin on the fly. Other times e.g. if a pure static analysis crashes it just logs the error and uploads it for analysis. That particular analysis might not appear as a consequence but there are so many different optional quality-of-life analyses being done you often hardly notice.