"You are arguing that storing the public and private key in a vault under the ocean is not less secure than just storing the private key there. While that is technically true, it's also missing the point."

Consider someone learning their way around this stuff, but not yet well versed. They've heard people on HN talking about how "storing public key and private key together" is bad. Would you recommend they 1) move .ssh/id_dsa.pub to a different directory, or 2) make sure .ssh isn't world-readable? The issue was insecure storage of the private key - presence or absence of the public key is entirely irrelevant.

I'd recommend they not store the private key next to the public key; that is, they shouldn't do "scp .ssh/id_dsa* remoteserver:"

It doesn't matter whether there's something stored with the private key, both because that location should be secure and because the private key can used to recover the public key.

Is the issue that you somehow believe "stored with" is not a commutative relationship?

Edited to add:

"It doesn't matter whether there's something stored with the private key, both because that location should be secure and because the private key can used to recover the public key."

Right, it doesn't matter in what should be the overwhelming majority of cases where "the public and private key are stored in the same directory" so it's a piss-poor test for whether things are obviously being done horribly wrong.