The other person who replied to this seems to have gotten up on the wrong side of the bed, but the core of their suggestion is one I’d likewise recommend. I’m doing the yubikey local auth using the yubikey’s PIV applet, which holds standard x509 certs that can be used via pkcs11.

You should be able to set up PAM to use them in that way, without needing any yubikey-specific hijinks. Something like https://discourse.ubuntu.com/t/smart-card-authentication/260...