You could run IRC server on whichever port you like and many servers do, the biggest benefit of matrix is that it's decentralized and easy for non technical people to engage in, no need for znc bouncers and things like that.