This is mostly due to the different nature of the subjects, though - CS enrollment fluctuates hugely with tech fashions, and includes people looking to get ahead in their job, people learning "how computers work", etc. Math, on the other hand, can stay to a more strictly academic curriculum, especially with their honors track.
I forget where I read this (I think something of Alan Kay's), but I like the thought that CS will go the path of biology, on the undergrad level, by breaking up into several fields. Just as "pure" biology majors are separate from the school of pharmacy, "pure" CS majors needs a different focus than future network technicians and Blub programmers.
You're absolutely right, I should have been more specific - I was talking about the required classes, which are mostly taught in large lecture halls (Intro, Algorithms, OS). Another great advantage is how accessible and helpful the professors are, esp. in the smaller classes you mention.