You are talking about "flat-volumes", I also recommend disabling it. Even forgetting the issue you mentionned, it just makes volumes harder to handle and reason about.
I wonder if they would have ever implemented this "feature" if it wasn't the default (and only) behavior on Windows...
Oh! When all of the GUIs say sound is at max, but it's either really quiet or silent, and it turns out that it's because even though the PA volume is up, somehow ALSA (which PA ultimately uses on the backend) got muted. But this is a PA box, so you have no GUI tools to set the ALSA volume!
Do wonder how much future hearing loss it has created by sudden spikes in volume...