Most people aren't walking around with one time phone numbers, they have a phone number that's shared by family, friends and co-workers that will consistently resolve to the same individual whenever someone wants to connect.

Being a unique number that is tied to a single individual, it can function as a proxy for identity. This, obviously, assumes you are operating like the average user.