Re:Chroma – A General purpose syntax highlighter in Go
by GrayShade on Sunday September 24, 2017 @02:02PM
(#15325753)
> There's ongoing work to use a layer on top of Rust's super-fast DFA-based regex engine to accelerate these grammars
Will it really be faster? It didn't seem so from that GitHub thread.
As a data point for anyone curious, I'm using Syntect myself in a toy project. With Oniguruma (C NFA regexes), it highlights 200 lines of Rust in 40 ms on a 10 W TDP Celeron, which is all right, but a bit slower than I expected.
Re:Chroma – A General purpose syntax highlighter in Go
by trishume on Sunday September 24, 2017 @03:51PM
(#15326204)
It'll hopefully be faster. The Rust regex engine is quite fast, I haven't done any profiling to figure out why performance was the same in my initial test. There might be something easy to fix.
It's definitely possible to get better performance out of the underlying model, since Sublime does, but they have a custom DFA-based engine that can test regexes in parallel with captures, which Rust's regex engine can't.
The more modern grammars are written with a regex style that makes heavier use of the stack machine and so only uses regexes that can be turned into a DFA. There's ongoing work to use a layer on top of Rust's super-fast DFA-based regex engine to accelerate these grammars https://github.com/trishume/syntect/pull/34.
The problem with using non-regex based grammars is that you have to write them yourself. Syntect is something like 4000 lines of code but the grammars it uses total around 35k lines, and that's just the included ones, not the full ecosystem of online grammars. Basically unless you only want to support a small set of languages, a non-regex-based highlighting library is fairly infeasibly for a single hobbyist.