Rust's JSON serializer is really fast (~300 MB/s). http://erickt.github.io/blog/2014/12/13/performance-digressi...

I'm imaging more along the lines of serializing editor state with each key press, encoding it, sending it to another process, having that process unserialize it, lex it, encode all of the cascading token changes and information about those tokens back into JSON, sending it back to the editor process which then has to decode the JSON and apply all of the token information to the in-memory representation as real overhead.

It's great that Rust will decode and encode the JSON quickly (I've been keeping an eye on Rust's MSVC progress before I consider using it for a real-world project), but the project I work on cares about optimizations on the level of in-memory data layout and memory allocations. Moving syntax highlighting into a pluggable system would be a huge performance regression.

> I'm imaging more along the lines of serializing editor state with each key press, encoding it, sending it to another process, having that process unserialize it, lex it, encode all of the cascading token changes and information about those tokens back into JSON, sending it back to the editor process which then has to decode the JSON and apply all of the token information to the in-memory representation as real overhead.

The latency of both Unix IPC calls and serialization/deserialization of small amounts of JSON should be measured in single- to double-digit microseconds. A frame at 60fps is 16,000 microseconds, so if this is causing even a single dropped frame, something is seriously wrong.

The lexing itself might or might not be a performance worry, depending on the architecture of the whole system, but that's no different from the lexing built into the editor (except perhaps in that people writing editors tend to care more about latency)...

(Well, this seems academic, since according to other comments RLS doesn't aim to put syntax highlighting out of process.)

There was a talk at RustConf by Raph Levien about his editor, Xi. It almost entirely uses JSON-RPC to communicate through a distributed architecture, and next-level speed is one of his design goals: https://confreaks.tv/videos/rustconf2016-a-modern-editor-bui...

> I've been keeping an eye on Rust's MSVC progress before I consider using it for a real-world project

The latest builds of MSVC claim to have native Rust syntax highlighting and autocomplete, I have not yet tried it myself: https://www.visualstudio.com/news/releasenotes/vs15-relnotes

I'm referring to the MSVC "backend" for Rust, not Visual Studio.

Ah! What things are you looking for in its maturity? In some ways, it's actually the most mature backend; stack probes work with it, for example. Windows isn't my personal area of expertise though.

I've been following https://github.com/rust-lang/rfcs/issues/1061 for a while. It seems that high-level pieces are there, but that certain elements useful in day-to-day software engineering still need some more time.

Yay, someone actually follows that issue.

If you have the time, you should really consider helping to improve the Rust experience on Windows. There's a huge shortage of Windows developers in the Rust community.

Unfortunately I am already overcommitted with open source projects. For now I am content to wait and see how it all plays out. It will be great to have another language where Windows support is really first-rate.

Seconded, and you have done so much work on this front. It'd be wonderful if we had more people to advocate for and help do this work.

Probably the biggest drawback of the MSVC backend is that its debugging symbols only have line numbers, no variable/type information. The rest is mostly Windows-specific issues like manifests and subsystems.

Actually, the PDB debug info has variable and type information and a lot of it actually works. There's just a few minor issues here and there with stuff like fat pointers.

Oh, that's fantastic- how recent is this? Last time I tried debugging an -msvc binary was only a couple months ago.

It might have been due to either https://github.com/rust-lang/rust/pull/31319 or an LLVM upgrade. But stuff like looking at the values of basic integer types, the fields of structs, tuple fields, thin pointers, and even many enums works fine. You just have to make sure you have debug info enabled.

Keep in mind std in the Rust distribution was built without debuginfo, so if you step into functions from that, things don't work too well.

MSVC is commonly used to refer to Visual Studio, more specifically the C++ compiler that's part of it (originally "Microsoft Visual C++"), which would explain the parent confusing the two.

Perhaps I'm not understanding you, but how does the speed of syntax highlighting (or, more generally, the editor) affect your project's performance? I'm trying and failing to imagine some sort of realtime system controlled by someone coding away.

(Also, you wouldn't necessarily need to transmit the whole editor state--just changes).

I work on Sublime Text.

Haha, I found this exchange pretty funny.

"My project depends on my editor being super fast."

"What are you writing that has that requirement?"

"I'm writing my editor."

Syntax highlighting is (usually) done by the editor, not the server in the LS protocol. The changes sent from editor to server are pretty minimal, and so the whole thing is pretty quick. The overhead, never caused us any problems.

The syntax highlighting is done in process using a declarative format, eg: https://github.com/Microsoft/vscode/blob/master/extensions/g....

I was referring to the paragraph about Swift and Source kit. To my knowledge, the majority of editors do not do syntax highlighting outside of the main editor process.

Oh I see. Yes that does seem prohibitively slow.