Yeah, that’s exactly it! Unfortunately the different skill sets required here are so different. You part need deep understanding of the low level inner workings, and part need great attention to detail in user interface design. It would probably be easiest to pull of as a team effort.
Finding the time to do that is a challenge though, particularly with the higher activation energy of having to learn the ins and outs of X11, Wayland, and other bits and pieces involved in building a functional *nix desktop.