They leak metadata to the operators of the server. So does Signal, albeit anchored to SGX nodes they pretend they cannot access. Signal also has phone numbers making even worse.

With matrix at least you can pick a server operator you trust to provide your metadata to, or host a server yourself.