For sure, though at the very least the examples you gave are mostly other social networks. So at worst, it allows them to know what other social networks you use.

What you want to avoid is using your phone number when doing things like online shopping, since that's when more personal details about you can be connected to your number, and therefore to the other social networks you used that number with.