I do not remember anything particularly difficult to get X11 up and running remotely, other than the XAuth stuff which was annoying enough in hardened environments, although those have not been commonplace, and X11 tunnelling via ssh has automagically taken care of XAuth.

The ultimate deal breaker, however, is X11 not being proxied by corporate firewalls, even though the protocol is extensively documented and the source is available. But that is because RDP has won over, and demand for X11 has diminished.

But, yes, X11 prefers local networks with flat topologies, and the protocol is chatty (again, not as big of an issue anymore unless run over a mobile connection with a high latency, but the X11 protocol compression helps with that).

What was so complicated about configuring an environment variable, and investing into good tooling like Hummingbird?

It was never as simple as configuring an environment variable.