Smartcards e.g Yubikeys help with this. You store the key on the smartcard, then use ssh-agent to have the smartcard perform whatever cryptographic operations are necessary. On Yubikeys at least, the keys can never leave the smartcard, by design.

You should encrypt your private key with a password. Then you can use ssh-agent to hold a copy of that decrypted key in-memory so that you don't have to type your password in every time you 'git push'

Is this actually an extra layer of security? Is it harder to get read access to another user's file with 0600 permissions, or to read memory from another user's process? In my limited understanding they seem equivalent (at least on Linux).

They're approximately equivalent on Linux. Adding a password helps if you accidentally tweet or email your private key, leave it on a flash drive, leave your computer's disk decrypted, etc.

It's primarily targeting offline attacks ie: I've stolen your laptop or I am accessing it when you haven't decrypted it.

But there are also niche situations where an attacker has read permissions for files but not memory. It's atypical, but in cases where you have 'confused deputies' it happens - for example, if I have a service that attempts to safely broker reads to files but that service has a logic bug where a client can trick it into reading sensitive files.

The classic example of this is a path traversal in a web service.

Also, while there's no boundary, attackers are a lot happier reading files than memory. They're only human.

In general I find that, when you can, you should just encrypt your data. It can protect you in some surprising ways.

Encrypting the file, or better whole disk encryption, prevents someone stealing the laptop and yanking the drive to get access to everything.

No, you are supposed to use a passphrase to protect it.

You should always protect your private key(s) with a password.