We use diffserv for this, for apps willing to use it. Example: ssh sets the imm diffserv bit for interactive use. cake respects that (I've cited the relevant paper elsewhere, another place is https://www.bufferbloat.net/projects/codel/wiki/CakeTechnica... but after extensive testing we settled on 3, rather than four tiers of priority)
stuff derived from the sqm-scripts use the same method (using htb + fq_codel) but the problem has always been that diffserv is not respected end to end. However, within your network, you can make your intention known and have it work, if you have the bottleneck.
Also, we have always made the latency/bandwidth tradeoff explicit - if you want less latency, you must want less bandwidth. It's the only safe answer to apps gaming the diffserv markings.
there's an awful lot of lit on FQ, what we do with fq_codel is to not only interleave packets better but apply congestion control signals at the right time so competing tcp flows don't overwhelm the link (with under 10ms of buffering (v seconds common on fifo ISP links)).
https://en.wikipedia.org/wiki/Fair_queuing
Of course, being perfectly fair to flows is sometimes undesirable, but making something strictly higher priority[1] is fraught with peril as you end up with a classification nightmare.
Having fq gives you the best shot at smaller flows completing sooner, and of big flows sharing better with each other.
Having vastly reduced buffering improves the responsiveness of competing TCP flows a lot, grabbing bandwidth whenever available, faster.
My take on folk that want "prioritization" is ask them to try some variant of sqm with just fq and codel and get back to us. being fair with well managed buffers works really well.[2]
[1] making something lower priority than best effort is actually a good idea. [2] but if you really want some flows or devices prioritized, see the sch_cake work mentioned on this thread. I still tend to think per host FQ is what many want rather than attempting to raise the priority of certain flows from certain services.
https://arxiv.org/abs/1804.07617