Just a note: if you do change it, keep it below 1024. Otherwise if ssh dies anyone on the server can create a client listening on that port and steal credentials.