I suspect that any remaining user frustrations are either issues with fundamentals of the design, impossible to fix without frustrating other users, or don't rise up to the level of a bug report, let alone a patch.

Based on the timeline of CVS, I doubt there are that many large performance issues that can be fixed without significant risk of breaking. In my experience, CVS was primarily limited by network and disk I/O, both if which ate generally much improved since the time of active CVS development.

Keep in mind that the effective scope of CVS is also shrinking as many users move on to other software; that means any issues are less likely to surface.