> For example, querying your compiler for a list of methods that apply to the current object, or a list of functions that start with “Foo” are mostly moving to external processes using LSP as the communication protocol.

That's why we have lsp-bridge and lsp-mode emacs fork :) Both of which build some infrastructure to avoid doing communication work with lsp-mode work in main emacs thread. So, heavy emacs users are building some async machinery which wraps another already async and relatively lightweight protocol, because core emacs facilities can't keep up with it. Architecturally it is kind of insane.

I think, lsp-mode fork is doing the right thing (from practical POV; it goes against "emacs is just an elisp interpreter" ideology though) and hope it gets into core at some point. A better solution would have being having first class async and background threads support at the elisp level. Which would never happen due to elisp messiness.

https://github.com/emacs-lsp/emacs https://github.com/manateelazycat/lsp-bridge

Your response reminded me of an Intel project about regular expressions. I think it’s called hyperscan, but I’m not sure. It’s a regex engine (I think that’s the correct terminology) that makes dedicated use of very specific Intel instructions to reach incredible regex speeds on Intel processors. Would be cool if Emacs could make use of this tech when configured properly. I bet there are some internals that rely on regex.