How do you think the Signal servers send data to your app if they don’t know, at least temporarily, the IP address of your device?

Whisper Systems say they don’t record/log that. But they are no more able to defy legal demands in their jurisdiction to do so than Proton Mail were.