Servers have control. I can quote the CEO of new vector in an argument we had about the insecurities of the protocol and what needs to be done to fix them where he said "good luck talking to your own federation." That reveals a lot.
A popular server controls the federation. It controls the de facto standard. If their server isn't the most popular they lose control. This is how they operate the standard -- on an ad hoc de facto basis. If they lose control they lose velocity and value as a for-profit enterprise. The matrix is all about control, just like the movie, ironically.
The protocol is deeply insecure. It's modeled on ideas robbed from blockchain Merkel trees and directed acyclic graphs for eventual consistency. But they don't use either properly. At the protocol level blocks in the chain are not identified by their hash, they're identified by arbitrary strings. The blocks have hashes but their implementation does not check the hash. Read that again. The DAG allows for arbitrary insertion of blocks into the chain.
Are these backdoors? Probably not. They're just idiots. And they don't give a shit unless, maybe, they're pressed to.
Regarding your insecurity claims, is there any particular area where Matrix has concrete weaknesses? Could you elaborate on that point?
As a sidenote: I really appreciate your work and think new homeserver implementations are invaluable.