You don't actually need to have X11 etc running to use headless mode, the libraries are just necessary for linking as there aren't separate headless and non-headless Firefox binaries shipped. The possibility of dynamically loading these libraries to remove this dependency is being looked into.