If you want to actually use the federating feature, you will probably join some big channels. And that causes pulling of some huge amount of data from other homeservers.

disclaimer: I tested it some time ago with Synapse. Now I see there is also new homeserver software, Dendrite. It is possible that it is order of magnitude less resource hungry, though I wouldn't count on that.

One of our primary goals on the Synapse last quarter was to make it possible for a new homeserver to join Matrix HQ (a large, public room) in under 1 GB of RAM. And we did it. https://matrix.org/blog/2021/06/15/synapse-1-36-0-released

It's not the slimmest beast, in part because Synapse needs to scale up to country-scale deployments, but its ability to scale down has significantly improved over the past year.

That said, Dendrite and Conduit (https://gitlab.com/famedly/conduit/) are exciting projects which will optimize for different operational contexts.

How and why is that different from a client being in the room? I'm pretty sure that neither desktop, nor especially phone clients are supposed to take gigabytes of mem to listen to chats.