You use the SSN (acting as the username [which links to authority to work or claim benefits]) combined with another piece of information that establishes your identity (acting as the password to establish that it's really you and not me who is applying).

For employment, the I-9 form is what we fill out as authorized US workers and to document same.

https://www.uscis.gov/i-9 (the "paper version" link works best)

You'll note that Social Security card is in List C "documents that establish employment authorization" and not in List B "documents that establish identity" nor in List A "documents that establish both identity and employment authorization"

I guess you can look at it as a form of off-line multi-factor authentication. You are using the Social Security card as proof of eligibility for the person whose identity you have already established using another factor. In that case, it's the card that acts as the factor, not the number (which if you've ever seen a social security card, it's ridiculous to think that it's in any way secure or even durable).