If you are going to use a password manager to store them there's no point in using different passphrases.
Also, the keys don't have to reside on the same device the password store is on. If you have one key per client device for example.
So to protect against that, don't use a password manager. Which means that you're more likely to re-use passphrases, because lots of them are hard to remember and keep straight. So breaking one gets access to many others.
My head spins.
Sure, in general, you want to avoid password reuse to isolate password compromises / security breaches (some companies are still storing passwords in plaintext in 2016, ugh). But in this case, we're talking about private keys that should never leave your local machine. If those are compromised (even passphrase-protected), then it's not inconceivable to think that your password store might also be compromised.
Yeah, reliazed that, my post was just my "logical" progression.
It is better to use a different passphrase for each key but it is also less convenient unless you're using a password manager (personally, I'm using KeePass)
> - How much less secure is it to not use a passphrase on a key?
That depends on the security of the computer where the keys are. I remember a Firefox vulnerability where one site exploiting it was looking for ssh keys on the local file system. So I'd say that a passphrase is very important.
> - Should you use a different key per user account, per server, or per use-case (i.e. personal or work)?
Personally I'm using a key per account per host.
> - Does increasing the amount of bits in a key really have an effect on the security of the key, or does it not make much difference in a real-world use?
Yes up until 2048 where the returns of increasing the amount of bits will start diminishing.[1]