Right, that's the way it should be done for ports that try to listen to non-local connections. That would actually strongly increase security, as non-privileged ports can be abused a lot too.