well, depending on what you do, you usually need access to the dri-device, the X11-driver and X11 (socket). This is relatively insecure I suppose because it exposes a giant attack-surface. Everything trying to mitigate this (short of running VNC) unfortunately still has some, but I would expect noone targets these broad-spectrum. An interesting thing to do would be to run with VGL and a quadro/...-GPU which allows partitioning into different rendering devices.

Haven't tried reading about this with wayland.