You're completely right that we have different goals to Signal (openness + freedom rather than privacy-at-all-costs), but I'm not sure that blaming federation is the right answer here. We just prioritised building an open standard over having E2EE from day 1, instead choosing to design things so we could add it later.
Again: I'm not dunking on Matrix. There are things the Matrix approach will do better than Signal can. But protecting individuals is probably never going to be one of them; you're competing with a project that has security and privacy as its overriding goal, and nobody at Signal ever has to ask whether federation --- a protocol complexifier if ever there was one --- merits a security hit.
I don't use Signal for everything (or even most things). I'm completely open to the argument that other messengers are better "overall" than Signal. But on a thread that asks the question of whether Matrix is as secure as Signal (not "secure enough", but rather "at parity with"), there is simply a clear answer.
That doesn't make Matrix bad. It makes Matrix different. Matrix has different goals than Signal, and those goals simply prioritize security and privacy differently than Signal does.