This is the line which remotely stores your contacts encrypted E.164-formatted phone numbers:

https://github.com/signalapp/Signal-Android/blob/main/app/sr...