Its end to end encrypted and ciphertexts aren't stored long term on Signal's servers, but Signal isn't truly p2p. To send a message a user encrypts the message using the keys for the user, sends them to Signal's servers to be queued, Signal delivers the encrypted message to the receiving user, the receiving user decrypts it. Phone calls would probably be the closest thing to p2p, but it still uses Signal's servers as a broker for connecting the two parties instead of some other form of discovery.

Signal also does not allow for third-party clients to connect to their hosted Signal servers. This further makes them an authoritative 3rd party, as you need to run the code approved by Signal to connect to Signal's servers to send messages to Signal users. Not that I am complaining about this structure, just sharing knowledge.