It's been a very long while since I played with any of this, and I don't want to rely too much on the widely understood to be lacking documentation, but are Xlib and XCB still not in a very weird situation in which one depends on another for different reasons? In this specific case, at least multiple years ago, glx wouldn't play nice without Xlib, if it would be possible at all without a lot of pain. I don't expect it to have changed for obvious reasons, XCB never being complete and X interest slowly waning.

https://xcb.freedesktop.org/opengl/#index3h1