There's tons of little details and inefficiencies in elisp that crop up such that easy things like "listen to responses" are hard to get fast. Random example: before native json support, large lsp responses (which can be multiple megabytes from some servers!) would take over a second and hitch the UI just to parse the json response. And that was after a lot of work to get response parsing dominated by json instead of GC time.
The only way an lsp-client could be slow is by making a request to the server and then blocking on the response - which is wrong and the package writer's fault because they are not using the asynchronous api emacs provides for communicating with a process.