Automated testing of video games is less painful than manual testing though.

Probably not what OP is doing, but there are way better ways to do that than VNC. You can adaptively stream gaming pretty effectively over any network especially a local wired one.

I wonder if Steam Home Streaming works for the Unity editor? It's a popular and accessible option.

You can literally dump frames out of a headless copy of a game into FFMPEG and transcode it into an adaptive stream.

I haven't tried this specific thing with video games needing live input, but you can run headless on AWS GPUs and dump those frames to an AWS Lambda process[1] to do the conversion and only have to manage the input. In this situation the only thing you are running is the game and not all the other bullshit that an operating system has to deal with in a remote desktop situation. All you would have to manage is the sync from the controller over a socket.

Another nice feature would be being able to save your gameplay and maybe edit it later for broadcast if you wanted.

[1] https://github.com/binoculars/aws-lambda-ffmpeg

It works for the whole desktop, so yes.