The reason is that scp was designed as a secure replacement for rcp, which was designed as a remote version of cp. The flags -p and -r were inherited from cp by rcp (see https://linux.die.net/man/1/rcp), so for compatibility scp had to use the same flags.

As for ssh, I haven't looked at its history, but I suspect it's older than scp; the intuitive flag for "port" is -p, so that's what ssh used.