It provides some security against attackers that aren't capable of subtlety. If yank your unencrypted hard drive out and walk away with it in my pocket I can't use your encrypted SSH key to impersonate you to other machines unless I can break the passphrase.