I think in practical terms that will end up being a toolkit/framework question. In many cases remote usability is already ruined under X simply due to clients doing wildly inefficient stuff, assuming MIT-SHM, or what have you. I've used one program where it was blatantly slow just going across a virtual switch to a VM on the same machine.