"the professors would have to dumb down the material taught in their classes to their less bright colleagues."

A fascinating side effect of open courseware and online lecture videos is the elimination of that peculiar historical belief. For example, per video evidence, the kids in Professor Strang's linear algebra classes are definitely not observably smarter than the kids I met in my linear algebra class. There is a small measurable quantitative difference, but in practice there is no observable qualitative difference. The much maligned concept of human biological differences observationally do not extend to mere ivy-attendance.