A hash of the phone number is as good as the phone number itself. Given a list of all phone numbers in use, it's trivial to build a rainbow table for them. And many you can also brute-force.

Actually Signal is uploading contacts with their first and lastname to the cloud now. Or is planning to do so. Read their blog about that f-ing PIN feature. Its explained there. I hope they don’t go through with it, I absolutely do not wish to use some cloud; not even Signal’s. My data should be 100% local. And that they’re gonna push this without a back-up feature for iOS feels a bit like them raising their middle finger to us.