If you end up with a giant ball of hidden complexity, that's a strong indication that you're dealing with that complexity at the wrong level.
Both PulseAudio and systemd seem to have the "throw everything out, start from a beautiful abstract interface and work down" mentality, which is a recipe for the mess that they've ended up as. Sometimes the best thing to do is pass the ugly complexity one layer up - which is, I think, one of the things that Unix does mostly right.
The root cause of all the complexity is (was?) the lack of standardization for hardware interfaces. The lack of some globally usable way of dealing with latency at the OS level is another issue. A thing like Pulseaudio is a natural reaction to the current situation in that it tries to both hide the complexity of the hardware interfaces and deals with the latency issue entirely in one program.
So the problem isn't Pulseaudio. The problem is the environment that media is forced to exist in.