Your account phone number, yes. But, most importantly, Signal doesn't store your social network in its servers in a way that it could give authorities the phone numbers of all the people you communicate with. Or, worse, the times and dates of those conversations.

So all it can tell authorities is that a person with x phone number uses Signal and still uses it.