Yeah, the GPU is a custom part not found elsewhere (IIRC the NV2A GPU is meant to be somewhat similar to the GeForce 2 and GeForce 3).

However, there is an emulator being worked on for the original Xbox called CxBx Reloaded, which is a continuation of the CxBx emulator. To give an idea of how far along it is, here's some recent footage of JSRF:

https://m.youtube.com/watch?v=8fYaMc18apI

Yeah, although not any more obscure than the GC/Wii GPU really.

I'd argue that the fact that the OG Xbox's homebrew scene used the illegally acquired official XDK for the mos part played a part. This meant that they didn't get as deep into documenting the hardware at that early stage, and instead relied on closed source libraries.

The same could be argued about the Dreamcast where there SDK was concerned but that has been emulated. Not perfectly but pretty well. I have read it was not just interest in the console but the GPU.

The GPU is way simpler on the Dreamcast since all T&L is done in software. It's GPU is just a screen space rasterizer.