Even on my local Linux system I wouldn't know how to get hold of the mouse data without using an X Windows API (or SDL on a console only app before X is run).
You may be interested in https://gitlab.com/kragen/bubbleos/blob/master/spikes/intell..., which reads from /dev/input/mice, although I haven't yet implemented support in https://gitlab.com/kragen/bubbleos/blob/master/yeso/yeso-fb.....
http://www.cs.cmu.edu/~412/lectures/2009-10-23_radeon.pdf
>>Provide 2D acceleration via the 3D engine.
a) tooling and familiarity
b) it's leveraged for modularity, composition and access control. Normal unix basically has no modularity and little composition beyond piping to stdin (which is of limited applicability). And of course no useful access control mechanism to speak of. The last 5 decades mostly just added layers of shitty hacks that don't work and that no one understands anymore (by contrast e.g. the original unix ownership and permissions model was misdesigned but at least possible to grasp).
Plan9 does substantially better in both regards via clever use of a proper union file system (which incidentally makes another horrible hack superfluous: symlinks). If you want your shell, or some other executable, to find some executable, you mount it into its /bin -- there is no $PATH. If you want some process or user to have access to a resource (file, device, server in the network, ...) you mount it into their filesystem. This gets rid of a lot of problems and special purpose solutions.
Interpreting bytes is message passing and this communication happens through the handle/descriptor to the file/object/whatever. As above, this handle could have come from the kernel, another process or the executing process itself (the process opened some file or whatever).
I think the message passing version happening through a handle can be more versatile as you could send this handle over to another process. Could copy it and distribute it to other processes in a easier way than injecting/changing a (presumably C ABI) into a process address space.
That means you can run graphical applications on the CPU server.
But lots of things in Plan9 present an interface that isn't just a single file. The 8½ window system, for example, presents not only /dev/mouse but also /dev/cons (character-oriented I/O), /dev/bitblt (to which you write requests for 2-D accelerated drawing operations), /dev/rcons (keystroke-oriented character I/O), and /dev/screen (the contents of the window—which is just a byte-addressable blob of bytes). http://doc.cat-v.org/plan_9/4th_edition/papers/812/ explains in more detail.
And, of course, file storage servers similarly provide an arbitrary tree of files, and when you remotely log into a CPU server, you mount your local filesystem on the CPU server so that processes running on the CPU server can access your local files transparently—including /dev/screen, /dev/mouse, and /dev/bitblt, if you so choose.