With a reaction to that I expected overwhelming technical objections to everything the parent said. Based on the actual contents of your objections, I think you would have been better starting it Yes! Additionally...

Think of it as security through irregularity instead of security through obscurity. You are not making your box secure by changing the ssh port, but you are decreasing the number of automated attempts that will hit your ssh port, which could, in theory, reduce your security risk.

> * Use a non-standard ssh port <= not security. plus i bet you're on 2222.

Right, marginally better than using port 22, but it's still something (better than non-action, if you will). I do hope one takes the vigilance to set a port number sufficiently high up, and sufficiently random looking. I recall something about nmap, that it scans only up to 1023 by default or something.

> * Fail2ban: useless for key auth, you don't remote brute keys like that

Still set it up -- for http and other services.

> If your comp can log directly without any authentication that is terrible practice. Your comp will eventually get compromised, stolen, etc and then its game over. The passphrase is no silver bullet but it helps a lot.

I agree... I emphasized seamless logging in because that seems to be a big point that Mosh is apparently selling itself on. I do have stronger authentication methods set up for the production machine. :)

Hopefully not over port 1023. If a user gets on the system and crashes your ssh daemon through OOM killer or one of many other methods, that non-root user can then restart its own daemon and listen to that port that you put >1023 'for security reasons'.

Accidentally answering yes when it says a new key was detected is all it takes to get keylogged.

If you have a Mac and secure your SSH keys with Keychain someone could take it, crack it open, freeze your memory, and dump it.

Two factor is definitely the best you could do on this front but who sets up two factor with SSH?