The X11 protocol itself[1] isn't actually that complicated: it's large and old, but the fundamentals are mostly consistent and the modern client bindings (libxcb) are even generated automatically from a machine-readable protocol spec[2]. Various extensions and standards composed on top of X11 further complicate things, but most of them are graceful client-side enhancements rather than required components of a functional X11 client and server setup.

[1]: https://www.x.org/releases/X11R7.5/doc/x11proto/proto.pdf

[2]: https://cgit.freedesktop.org/xcb/proto/plain/src/xcb.xsd