That's not entirely true. It's possible to assign source port/address combos that route to different hosts.
For example, I can say that:
* 1.1.1.1:80 routes to 192.168.1.2:80,
* 8.8.8.0/24:80 routes to 192.168.1.3:80, and
* everything else on port 80 routes to 192.168.1.4:8080
It's possible to get even more convoluted, though not recommended.
(If I read your post as written, it also seems to say that the client needs to pick a custom source port and if they're using NAT then they also need to get that source port unaltered through all of their layers of NAT. And they'll be limited to one connection at a time, perhaps one connection for however many people are behind the same CGNAT, because only one connection can use a specific source port at a time.)
So yes, you can pile hacks on top of hacks on v4, but it's not similar effort to just permitting the connection in a firewall, and it has a bunch of extra limitations too.