The processes-communicating-via-IPC architecture is way underrated for plugins. It's much easier to separate the plugin from the host, and to cleanly stop or restart the plugin. For example, sometimes (rarely) rust-analyzer hits a bug and starts consuming 100% CPU. When that happens, I just kill that process and VS Code transparently restarts rust-analyzer; I don't have to restart the IDE, and all other plugins are unaffected as well. That would be practically impossible if rust-analyzer were a library instead of an independent process.
It means that as a user of the system you must understand the inner workings of the system.
Imagine someone says this about a car.
"If you are driving on the highway and some component dies midway, you just open the cover and press a button on the broken component and it restarts itself! Isn't that much better than waiting two weeks for some repairman to repair it?"
I don't want a car that has components that just randomly die mid flight. That's not a feature. It's a deal breaker.
In more general terms, a system where failures are hard is easier to make robust because the errors are visible to developers.
In a dynamic system where failures are "not my problem", it's very easy for the system to develop into a wobbly unstable mess, because no one owns the system. It's just a bunch of plugins that may or may not behave erratically.
That's not a good way to build a stable relaible IDE.
Completely bug-free software is a desirable, but utopian goal. What do you think why processes with separate address spaces were invented? Would you rather go back to the Windows 3.1x days where a bug in a single program could bring down your entire system? Do you believe those programs were "easier to make robust because the errors [were] visible to developers"?
> "If you are driving on the highway and some component dies midway, you just open the cover and press a button on the broken component and it restarts itself! Isn't that much better than waiting two weeks for some repairman to repair it?"
Yes, that would be much, much better than waiting for two weeks while being stranded in the middle of nowhere.
How far would you extrapolate this? Where would you draw the line?
Would you say VS Code would be more resiliant and reliable if it had more isoldated processes for different parts of the editor?
- Process to blink the cursor
- Process to render the text
- Process to run syntax highlighting on the text
- Process to show the buttons on the left most side bar
- Process to draw the file/directory tree
I think it would be obvious that this would be insane.
Yet, why should it be? Wouldn't it be better - according to this way of thinking - because it means the editor will not crash if the cursor blinking code crashed! or if the file tree viewer crashed!
To where it makes sense from an engineering perspective. Yes, that's what makes good engineering: selecting a pragmatic point between two extremes.
> Yet, why should it be? Wouldn't it be better - according to this way of thinking - because it means the editor will not crash if the cursor blinking code crashed! or if the file tree viewer crashed!
Your argument is absurd. You're trying to counter my point by making up an extreme version of what I said and then refuting this imaginary point. This kind of logical fallacy is called "straw manning" [1]. This is how it works:
"According to your line of thinking, wouldn't it be better if every program shared the same address space, even in general purpose computing? Maybe even with the kernel? Then every programmer would be really* invested in avoiding bugs. I think it would be obvious that this would be insane. Therefore, you're wrong and I'm right."*
Can you see now why that kind of argument is a fallacy?
You didn't answer the question though.
That is just assumed within my question.
The question is where do you draw the line.
Why is it ok for the autocomplete to be a separate program (and somehow that's good because the autocomplete can crash without crashing the editor)? Why is it not ok for the syntax highlighter to be a separate process? or the file viewer?
> You're trying to counter my point by making up an extreme version of what I said and then refuting this imaginary point. This kind of logical fallacy is called "straw manning"
I'm taking your basic premise and showing you that if you extrapolate it to its natural conclusion you will end up with an absurd situation.
This is not a fallacy.
Although my general feeling is, when you start accusing people of being fallacious, it's difficult to continue the discussion in good faith.
Why is that?
1. An in-process type safe API can be evolved and iterated rapidly. There's no need to write informal English specs for everything because the API itself provides so much guidance. Actually, IntelliJ is rather notorious for its absence of JavaDocs though this reflects cultural decisions inside JetBrains itself more than anything intrinsic (they take "code should be self documenting" perhaps a bit too seriously). In contrast, the LSP spec is enormous and the document itself doesn't provide much assistance implementing it (the article discusses this somewhat).
2. A protocol spec doesn't give you any re-usable code. If you write an IntelliJ plugin there's tons of re-usable helper logic and you can easily customize other plugins. The whole thing is designed for on-the-fly presentation compilers. With LSP you may or may not have anything to base your work on.
3. An IDE is fundamentally a highly parallel mutable database, in which the data can be in arbitrarily invalid forms at any point and the user is allowed to change anything whilst many different analyses are running in parallel. Shared data structures are everything in this problem, yet the LSP model effectively forbids all shared state between backend and frontend. This feels like a constraint inherited from browsers - it arguable doesn't make things easier, it just replaces relatively simple and statically/dynamically analyzable locking protocols with complicated sync and rec protocols. Both sides have to maintain their own copies of each other's data structures instead of re-using a single source of truth. This can get really messy, really fast. You're also constantly paying the costs and overheads of multi-process like context switching (especially expensive on Windows), IPC copies, process overhead, serialization etc.
4. A presentation compiler doesn't seem to benefit much from being written in Rust or C++. Developer workstations are fast and have plenty of RAM, so GCd/JITd languages don't seem like a particular problem here. Developer productivity on the other hand is at a premium. People want features. One reason JetBrains can add new languages and features so fast is that they're writing in relatively forgiving languages like Java and Kotlin, where you don't have to think too hard about ownership or memory management. Some thought is still needed - see the disposable model inside their IDEs - but you can more or less just go for it with a high rate of feature throughput.
You raise the question of how to kill hung operations. In practice that doesn't seem to be a common problem. I've been using IntelliJ with various languages for years and there have been many bugs, but I can't remember any time that a plugin hard-hung whilst analyzing. There are cases where plugin bugs happen but exceptions and GC mean the IDE can simply catch the errors and disable it. These days it can do that temporarily and simply reload the plugin on the fly. Other times e.g. if a pure static analysis crashes it just logs the error and uploads it for analysis. That particular analysis might not appear as a consequence but there are so many different optional quality-of-life analyses being done you often hardly notice.
Huh? The first part of the sentence makes me think of Rust. It's fast, memory safe, type safe, and thread safe.
Is GC a problem in language analysis systems? Doesn't feel that way these days, and anyway JVMs have pauseless GCs anyway now.
In terms of speed, as long as you don't choose Python I think you'll be fine.
I do think in terms of productivity, something like F#, that has GC, ML style syntax with piping and partial application, and a strict enough type system with Hindley-Milner type inference, would be the most productive.
My hold up was mostly with the sentence I quoted which seems to contradict itself.
Microsoft didn't go that way and again it feels like maybe JS limitations are the cause. Calling into a native library is tough in JS because there's no standardized FFI (browsers wouldn't allow it) and you're going to end up blocking the one JS thread you've got. Native code is going to expect the host to be able to spin up threads and use them because that's a super reasonable assumption, but one V8 can't meet. So they want everything to look like a web server because that's what JavaScript is intended for. That pushes them down a particular path, but it's definitely worth stepping back and saying ... does this really make the most sense, overall?
That's basically what separate processes are, so what's your point?
> Calling into a native library is tough in JS because there's no standardized FFI
No need for using anything standardized: VS Code brings its own custom browser engine – for better or for worse – so it's not limited to what any Web standards allow or forbid.
> you're going to end up blocking the one JS thread you've got
There are worker threads in JS.
JS has workers but they aren't sharing memory, so they aren't really threads.
The "shared state" in this context is IDE program text, and analyzer diagnostics. Copying those across address spaces and keeping them in sync is super fast, the overhead is too low to matter.
Recall that most analysis is done async, so results arrive at different times. Even if you're not doing anything then, your IDE/backend will be context switching back and forth as stuff arrives to be rendered.
In the limit this degrades to the backend being the entire IDE and the frontend being just an translator between the OS IO system a stream of JSON rendering commands.
The way I see it, there are possible two grand architectures:
* you either have a single, multi-language combine supporting any language. In a sense, LLVM/GCC but for front-ends
* or you have a bunch of independent servers
The first is clearly less code — there’s a lot to share. The second is embarrassingly parallel though. So, if you are a (single-threaded company), you should go for the first approach. If you are a (multi-threaded) OSS community, the second approach has shorter critical path.
Some second order effects:
1. While backands are more or less the same between languages and LLVM is a clear win, frontends tend to differ a lot. That’s the point of having different programming languages. So in practice sharing the code in IDE is harder. 2. An interesting benefit of separate process architecture is that it forces you to have async interface, which is important for responsive IDEs. But that’s an argument for separating back and front, not necessary for sharding back.
This would mean that you'd have to write a compiler not just for each host language/source language combination, but for every IDE/source language combination, because Atom would have different internal data structures than VS Code.
Historically for IntelliJ that meant writing everything in Java. Nowadays plugins are written in many different languages. You could write plugins in Ruby or Python if you wanted to. They run on the JVM. You could even use C++ and get the same benefits because GraalVM can run C++ on the JVM itself, i.e. it will apply GC to the C++ parts so you don't have to worry about ownership across the interface. But that'd be an advanced move and I don't think anyone has tried that; for such big gaps in language they tend to fall back on the more conservative multi-process model again. Stuff like Sulong is too new to have made such an impact.
Atom would have different internal data structures than VS Code.
LSP doesn't solve that. It either forces everyone to implement the LSP model and data structures, or you have a mismatch and translation anyway.
Most issues with the VSCode plugins I have is that they don't report errors to the user properly. It's really hard to find out _why_ something is failing. So when everything is configured correctly, the LSP works great. When you have a misconfiguration you need to play detective.
Plenty of plugins hang forever, fail silently, or they give you very generic error messages. But this seems to be a VSCode issue.
That’s a protocol issue! While LSP has a method to report a transient (edge triggered) error, it doesn’t have a way to report a persistent (level triggered) status. And to solve configuration problem, you really want a “status” concept - you need an LSP health indicator which is either red or green, and which you can click on to get info about what the server thinks the project is. There’s no such feature in LSP at the moment.
https://github.com/microsoft/language-server-protocol/issues...
But even then, not all of them are great. For example, the Python plugin is of questionable quality, but that is probably because Python does not amend itself well to intellisense, even when you add type hints to the picture.
> Most issues with the VSCode plugins I have is that they don't report errors to the user properly. It's really hard to find out _why_ something is failing. So when everything is configured correctly, the LSP works great. When you have a misconfiguration you need to play detective.
Good observation. This is not just a "feature" of LSPs. It's a feature of all distributed systems developed with the unix mindset.
For example, Docker suffers the same problem. Someone gives you a docker image, it works great! For a while. But when something breaks, it really breaks, and it doesn't tell you why. It prints some superfluous error message; one that does not help you understand what's going on.
That is also a good observation. It's hard to create a good UX with a system where you don't somehow handle all of the failures of the subsystem in a central manner.
My problem here with VSCode is that it often doesn't even expose the failure of the subsystem, essentially not even giving you the information you need to find out the issue.
If my corporate proxy fails, the extension store will say "XHR failed", and before it used to just hang forever.
But as a user, what is XHR? Why does this hang forever now? There could be many reasons.
At least tell me that you had a failure while downloading something.
It's one of the reasons why the Linux UX will always be subpar.
Apple's macOS is based on unix underpinnings, but they take ownership of the whole thing and this is why they can beat it into shape.
So for me, the question is wrong. It's not "Why is LSP so great?".
The question should be "Why are people so hyped about tech that's based on questionable assumptions and that hasn't proved itself?"
The reason VSCode invented LSP is that different programming languages don't have a reliable way to intercommunicate (other than as separate processes communicating over a socket).