After a certain point the number of selectors does affect parsing time and selector matching time, in a very significant way.

Performance has improved a bit since this post but it's still relevant: http://perfectionkills.com/profiling-css-for-fun-and-profit-...