Probably because the people developing these clients aren't being bankrolled by multi-million dollar companies with private interests. The previous poster was correct; Matrix is a protocol, and so your performance really comes down to whatever client you're using it with. Element is a web app, so it will inevitably be pretty slow. If you want a client that won't slow down, look into Fractal, a GTK Matrix frontend written in Rust. If that doesn't iron out your performance issues, you just have a slow machine/connection.

Fractal is actually responsive, I am actually impressed. It's not bankrolled by multi-million dollar companies though and Telegram web app is way faster so something doesn't add.