I understand what you want but as someone who spends a non insignificant time watching youtube videos over what amounts to a glorified VNC (it's not VNC though) I don't see why a video compression based solution wouldn't work for you. The only downside is that real time video encoding is very CPU intensive if you're running it on a budget VPS. However, CPU based HEVC decoding is also really bad on a VPS. On a dedicated server or machine it's going to be insignificant though.

>If RDP worked like X, I'd love to setup a beefy windows server in my basement and play games on it via RDP from my PCs around the house. If anyone has tips on that, would love to read them.

Let's be honest. Using X for this would have absolutely atrocious results. Steam already has a solution for that and it is based on video encoding. You can stream games from a local machine to any computer with the Steam client installed on it. There is also dedicated hardware to stream to a TV and use a controller instead so you might not even need more than one computer. [0]

[0] https://store.steampowered.com/steamlink/about/