You can use Yubikeys(most of them, the cheapest ones don't have that feature IIRC) as smartcards in addition to other features.

You can use smartcard in linux as both GPG key, and SSH key (via GPG SSH agent).

Which means leaking your private key for SSH is essentially impossible, as key can be generated on device which means it never leaves it, even if your machine gets compromised