In a typical WebAuthn scenario the Private Key is actually being stored by the Relying Party, they just don't know that because it's encrypted with a symmetric key. That symmetric key is baked inside the hardware token.

Yes I'll say not being able to get that symmetric key out of the hardware token is a feature, because it is. Without that feature you'll have to educate users about how to care for their symmetric key and every time you inevitably fail they get exploited.