If you think about it in terms of network topology, you'll see it's not completely arbitrary. In fact, the long-term plan for Matrix is to explicitly support the fully decentralized topology by using Dendrite, where a client wouldn't need to connect to a dedicated server at all. Instead, each client is also a server.