Your fingerprint/faceid/whatever is used to access the passkey. It is not the passkey. To that end, yes, if you are worried about clandestine access to your phone (if that is your passkey), then you probably don't want to allow access using fingerprint/faceid. And if someone can copy your passkey off of your phone, you are again compromised.