Some tiling window managers work this way without any issues.

Bspwm[0], for instance, uses a client CLI program (bspc) to send commands to the daemon and alter its configuration at runtime.

In fact, it doesn't have a configuration file format at all, since it can be configured via a simple shell script that runs a sequence of bspc commands.

It also doesn't have any key binding functionality and offloads this to a separate program, sxhkd[1], whose only purpose is to interpret key presses and execute shell commands. Since this is bspwm's interface, the two work seamlessly together, even though they're not dependent on each other at all.

This is just brilliant design, and speaks volumes for the "do one thing and do it well" school of thought.

For the record, I've been using both programs without a single issue for the past 3 years at least, and it's been one of the most stable and least problematic systems I've ever used, and use to this day.

So just wanted to offer an opposing personal experience (anecdotal, to be sure) regarding your brittleness concerns with this interface.

[0]: https://github.com/baskerville/bspwm [1]: https://github.com/baskerville/sxhkd