If you change your SSH port on your Linux machine you might be misidentified as Windows machine, because these usually does not have SSH, thus next step will go for RDP. Nothing there either.

Sure next step can be going for a port scan, but how big scan do you want to do before fail2ban or similar will lock you out?