Instead of 'cutting out' a part of your screen to export via VNC/RTP, you could use EVDI to create a virtual DisplayOutput, that can be bound to one of the available rendering heads of your graphics card (so with HW-Accel), and then exported either through VNC or RTP. That way you can extend your desktop onto nearly any external display surface.