Yeah because if attackers are already finding a way in, they can easily read my ~/.ssh/id_ed25519. A password is strictly better, although not by much because lateral movement is very easy. The only exception is if the private key is actually on a HSM, which is the case for all my SSH private keys on iOS.