A motivated attacker will always tend to try the "low tech" approaches first to rule them out as a first step because it it doesn't require as much sophisticated technical attention and in addition it's a well known fact that human beings are more often than not the weakest link in the chain of security.
It goes something like this:
1. enumerate the employees.. in other words build a list of all their work emails and try to obtain their personal emails and those of their spouses and children.
2. investigate the background of executives so that a compelling phishing email message can be crafted which looks like it originated from high up within the organization.
3. Send targeted phishing emails designed to bait people within the organization to visit webpages that exploit Adobe Flash attack vectors or other browser based vulnerabilities, or perhaps even lure them into installing a trojan directly.
4. after someone in the organization falls victim to the client-side attack, read their emails to learn more about the organization's structure so that your next phising attack can be more refined, specific, and compelling and can possibly utilize a real corporate email address.
5. Rinse and repeat until you eventually gain access to a developer laptop where you can grab production environment SSH keys. Now you own the network without even having to scan a single server and without leaving a trace in the log files.
For startups, the #1 risk is a vulnerability in some web app, or a forgotten admin panel, that leaks the entire database to an attacker. Not a complex attack conducted by a nation-sponsored offensive team.
But again, SSH security matters, you should take it seriously.
I think that you may have a dangerous attitude about it because in modern times it's not a question of whether or not you're a big enterprise or a startup it's a question of whether or not the dataset at the nucleus of your system would be valuable on the black market or not. If an attacker or group of attackers thinks that your dataset could be saleable one day in the future as your company continues to grow then instead of trying to buy your equity on the secondary markets they may invest in trying to "own" your infrastructure now before you become big enough to put your employees thru white-hat training around social engineering.
I suppose my point here is that it does make sense for startups to put their team through proper white-hat training but it doesn't have to be expensive because you can roll your own. What I suspect is that in 10 years or so this kind of anti-social engineering training will be a standard for any IT knowledge workers not just programmers and will likely be part of the job interview process.
We are aruging over something moot though, since we both agree.. take it seriously.
It's also the least likely to happen. Attackers will have broken deep into your database due to poor webapp security before getting a hold of your ssh private key.
Not that SSH security doesn't matter. It's easy enough and carries enough risks that putting some effort into it is worthwhile.