There have been the occasional Inkscape builds over the years which use the native macOS er.. window manager instead of needing an X server.

Saying that because I used to use them, and they worked fine (for me).

If you're ok with using a slightly older Inkscape release, you should be able to dig up a few builds that work. ;)