You're absolutely correct, but it's actually pretty common to distribute that public key all over hell and then start shuffling the private one around between the ones you find yourself ssh-ing off of. At my last job, this practice among wayward sysadmins caused me a lot of gnashing of teeth.

Wrong move though - really there should be one key per machine - and then ssh-agent set up properly to take care of the intermediate machines so you can bounce around. Using the same private key on multiple machines is just asking for trouble when you need to revoke something...

Oh, believe me, I know. That's why I said it caused me much gnashing of teeth. SSH, particularly OpenSSH, is such a thing of beauty. It's a shame most sysads that I've dealt with abuse it like it's rsh and telnet.

I'd agree that uni-directional transfer is an edge case when you have only one server.

Unfortunately for network security, that seems less and less the case these days. I definitely find myself ssh'd into two remote systems relatively often and have a need to copy files between the two. (Swapping arguments to scp is not good enough.)

Doing so, you expose yourself in propagating penetration from a compromised server to a previously secure one.