IMO you don't really need it to be a full vnc style remote desktop although, or even have the editor run in the browser. You can get equivalent results with bazel[0] remote execution + cache servers and get a similar horizontally scaling build system, without vnc style jank or full network dependence for your actions as a developer.

Another reason why google likes the remote dev experience is because it doesn't download code to the developers laptop, because they don't trust them.

[0] yes i know bazel / blaze is made by google