Re:Emacs is special regarding UIs
by Spivak on Thursday September 10, 2020 @04:41PM
(#24436378)
I hesitate to call it an official model but the short version of the current direction is that the compositor acts as the glue -- providing the final rendered scene and the final merged audio stream to the client over some protocol (like VNC but doesn't have to be) and providing inputs and events to itself and the client applications.
How all of this works on the backend is a combination of PipeWire to multiplex video sources and PulseAudio to multiplex audio sources.
> The main difficulty in producing such a tool is that Wayland protocol messages primarily include control information, and the large data transfers of graphical applications are implemented through shared memory. waypipe must then identify and serialize changes to the shared memory buffers into messages to be transferred over a socket.
How all of this works on the backend is a combination of PipeWire to multiplex video sources and PulseAudio to multiplex audio sources.
There's nothing outright preventing something like X forwarding and https://mstoeckl.com/notes/gsoc/blog.html is a really cool project that makes it work.
From that author:
> The main difficulty in producing such a tool is that Wayland protocol messages primarily include control information, and the large data transfers of graphical applications are implemented through shared memory. waypipe must then identify and serialize changes to the shared memory buffers into messages to be transferred over a socket.