For anyone else reading, it's worth noting that these issues are with Synapse, the "reference implementation" homeserver built by the team developing the Matrix spec. Performance issues are a focus for its development now. There are also other projects (like my project Ruma) that provide alternative implementations that won't necessarily have these performance issues (hopefully!) The nice part about having a spec is that anyone has the opportunity to provide a superior implementation.