Yeah, that's the annoying part. Been wondering about this for years, and graphics support was among the first issues raised on the lone lisp GitHub repository. At this point I've even started exploring the mesa codebase, made some patches but didn't submit them yet due to the AI stigma.

With Linux system calls alone it should be possible to set up kernel mode setting without depending on any toolkit at all. This should be enough to get a framebuffer for software rendering.

For hardware acceleration though, one must give this graphics context to an OpenGL ES implementation. That's where it gets ugly. There is no way to divorce that from the libc short of literally rewriting it.

Maybe Vulkan will enable it? I can't say for sure at my current knowledge level.