To add to this a bit, another area where the "X ecosystem" didn't keep up is with toolkits. X11 is a very asynchronous protocol, but this is difficult to work with for toolkits so they synchronize most things, making it slow over high latency links (but unnoticeable over low latency links, like UNIX domain sockets on the local host).