That mirrors my experience with the matrix.org homeserver. It has not improved much over all the months.
We've been working away fixing causes of 'unable to decrypt' bugs, to the extent that (as a poweruser) I almost never see them. If you see them, please report them via bug report so we can dig into them.
Meanwhile, the performance of the matrix.org server has definitely improved massively over the last few weeks. We hit a performance ceiling from May through mid July, but since July 19th or so we've finally got CPU headroom back again thanks to stuff like https://twitter.com/matrixdotorg/status/1019957885026144257 and https://twitter.com/matrixdotorg/status/1022095383978233856.
Yes, we had one Synapse server running on a resource-constrained machine that sometimes "fell behind" the rest of the network. I believe that is what had caused such issues. Still, the fact things easily break with server load or network issues means there is something faulty about the protocol. Resilience and reliability are no less important than security.