I understand the mechanics, but from a usability perspective it makes no sense that you can uninstall the interface of the OS you are using just by uninstalling a language dependency.

"The interface of the OS"--what interface? I promise you I'm not being pedantic. The OS doesn't/can't know what "the interface" is for you. Yours, to that box, is Xorg. My interface to all but two of my Linux boxes is SSH/zsh. Somebody else's might be SSH/bash. Changing the package management tools such that I have to do something nonstandard within my configuration management tools on the off chance that you fire a footgun is pretty lame, yeah? (Ditto the reverse. If I uninstall zsh, I expect something to break next time I try to SSH in.)

Choices imply the ability to make a bad one. And it's a fairly recoverable one in your case, so I gotta say the criticism feels weak.

No, I'm not. I'm telling you that there are many interfaces and that expecting generalized tools to understand your specific use case out-of-the-box is unreasonable.