1) Remote computing doesn't have to use X11. The local machine can do some of the lifting. Wolfram Alpha on mobile phones is a good example of this- a server does the crunching, but neither is the client side a dumb terminal.
2) I cross my fingers that one day latency will be prioritized more... IMO the majority of latency problems are not caused by the limits of physics, but rather the simple fact that not many people care about it.
The theoretical peak ping speed across the USA & back is something like 20ms, an entirely reasonable figure for remote computing- if only it was ever anywhere close to realized.
If you look at network QoS research in the past 20 years, you'll see that a lot of people do care. It's just not an easy problem to solve.