Some homeservers are larger than others (e.g. matrix.org). They don't all need to be compromised to enable mass surveillance. It also depends on where TLS is terminated. If you're running a homeserver on AWS or something behind their load balancer, there's a difference.

Generally, I'd argue that E2EE provides defense in depth against "unknown unknowns" if server infrastructure is compromised by any means. Although I do acknowledge it adds one more level of complexity, and often another 3rd party dependency (presuming you're not going to roll your own crypto), so it's not a strict positive.