Correct me if I'm wrong or missing something, but doesn't the use of SSH resident keys still require a file to be present on the client before it can authenticate? I'm prompted to use the SK to prove my 'presence' after the standard identity secret keyfile is checked.

To be clear, my goal is to simply plug in my SK to a fresh OS install and "magically" be able to SSH into my servers.

Maybe not “magic” but you can get very close to that with “Discoverable Credentials” on a FIDO2 key.

The process for using a discoverable key on a new machine re-imports the relevant public key and private key handle to the new machine when you “ssh-keygen -K”. Its roughly equivalent to copying key material around with a flash drive, but without the need to remember two physical items.

That’s how it works for me. I generated a new key on the device, installed the public key where appropriate, and tap to authenticate.

https://support.apple.com/en-us/HT208372

https://playbooks.idmanagement.gov/piv/engineer/ssh/