This is a whole post, but basically, there are multiple SSH agents. ssh-agent supports the card and ed25519 keys, but doesn't support persisting keys across reboots. Gnome keyring supports persisting keys, but no card or ed25519 (AFAIK). gpg-agent supports persisting keys and ed25519, but no card.
Unfortunately, there's no perfect solution, so I just added an alias "yubissh" to include the library in the command line :(
How did you solve it?