Hardly any of the use cases I have heard for networked graphics actually require or even care where the rendering occurs. They are just concerned with application software installed and running in one place causing pixels to appear for a user elsewhere. These could work just fine with remote display streams, with all rendering happening on the same host where the application executes.
I ponder what it is that I really dislike about VNC or RDP, and I find it is the nested desktop and lack of integrated window management. I think I'd be fine with some kind of "rootless" variant to let me SSH-forward a set of windows to a Wayland compositor. I would prefer it to have more flavor of tmux/screen to let me connect and disconnect, and easily define named sessions or groups of windows that I would want to grab together.
They haven’t removed it from X have they? You can keep running X if you want to (you may need to start maintaining it yourself.)
They don’t use it so they don’t want to implement or maintain it. Seems super reasonable to me?
There's also remoteapp which allows you to just run one program remotely, but haven't got much experience with this.
Last time I checked it was a X extension, if at all.
What does an optimization have to with anything? So what if X does not have a native GPGPU driver, that's really more a function of hardware manufacturers support than anything else. And if you are not too picky about all your stuff being 'open' then NVidia's X driver and CUDA on linux co-exist just fine with accelerated graphics and GPGPU support.
RDP is more closely related to VNC and something like 'screen' than X, which is more like a networked resource that you can access through remote clients.
RDP is an application sharing protocol, X is a screen sharing protocol, the two are radically different from each other which gives each advantages and disadvantages that the other lacks and a bunch of overlap. Most notably, with X security was bolted on as an afterthought. RDP, originally reading the screen contents and sending those over (compressed) does not allow for much interaction with what gets sent over, it is rather low level whereas X sends over display primitives. Again, X's initial - rather naive - implementation made a ton of assumptions because the original implementors had nice fat workstations and fast networks to work with and so the real-world utility of these features was rather limited.
Anyway, you could write books about this and not get all the details, besides the various implementations of both but all that there is to this is that due to their history and intended application the two are very different beasts and X offers a versatility that not a whole lot of people need but when you need that versatility you need it badly.
Per a discussion the other day, my RPis are in a drawer. Have we swapped out screwdrivers for RPis in the modern junk drawer?
No, they've been added to the screwdrivers. I've found it challenging to use RPi to drive screws.
The Unix philosophy: do one thing, and do it well. If you want a remote renderer, write a piece of software that does just that, specifically.
Yes, this means that unlike many C-S architectures, the server is the part nearest the user and the client (may be) further away. But each did its respective job and stuck to its knitting.
Keep in mind that when X11 was first created, and for much of its early life, through the 1990s, it was used for remotely accessing another (or multiple other) systems. The networking capability was baked-in. Nevermind that security was bolted on later, here, have a MAGIC_COOKIE....
You're trying to argue that there should be two distinct pieces of software, a local and a remote renderer, for simplicity.
Rendering does not fit in that space well. The physical resource of a display and graphics card really only allows one renderer to run, so it's not really useful to say "just write two renderers, one local and one remote".
The unix philosophy is not a good argument. It's a principal by which you can make decisions, but it doesn't always lead to good decisions and it's quite open to interpretation as well.
Unix is different things to different people and to remove the network capability from X is a step back, not a step forward. There are a whole pile of neat things that you can do with networked graphics, so much, in fact that Plan 9, in many ways a successor of Unix used that theme as the core of many of the system services. All of them work locally as well as remote.