That is so true. Github had a lot of performance issues in their diff pages and they solved by using one-to-one mapping class-names and keeping the specificity as low as possible. Here's more on that: https://speakerdeck.com/jonrohan/githubs-css-performance?sli...
If your CSS rules are applied to classes, it will gather the classes first then filter it down (Resulting in a smaller set in the first place).
CSS processing will be much faster.