The whole point is in avoiding starting with an AOL like service. So far only big matrix provider are reliable and performant enough to be usable. This is @gmail.com all over again but with @matrix.org tld.
Except you won't be able to carry your messages from a tld to another when you decide to rely on another domain name (your own or someone else's).
How long before Matrix foundation send messages telling users they are going to delete their rooms and messages if they don't log in once a year ? Or that they are now restricted your account to matrix.org rooms to "save operating costs" ?
The whole tech stack is free but operating costs are not.
I've been running a Matrix homeserver on a 1/1 VM for years without any issues. There is no downside to choosing a small server, you can still federate with everyone else. That's the entire point.
Start on a server, but your real identity is attached to a cryptographic key, not an e-mail-like identifier. That would allow you to move around, and maybe one day get rid of domain names altogether (using something like yggdrasil or tor to host and connect servers, for instance).
Signal offers no such choice.
Even if you don't do this, you can still reach contacts on the old server and middle through.
If you switch from walled garden to walled garden like WhatsApp to signal there is no migration path at all.
Maybe you started on AOL and later realized AOL is terrible. You could export your address book and move to a client/server you trust more and notify all your contacts from the new location.
This is the same story on Matrix and what I mean when I say it is a freedom respecting decentralized service.
You are also free to run your own DNS to a dedicated EMS instance then later point to your own self hosted server later much like the freedom you have using your own domain and MX records on Google Apps allowing you to later move to a new email provider without having to update your social graph to change your address.
On Signal, there is no such option. You use their clients and servers forever, or GTFO.