Yes, true. Still, our society encourages innovation, where anyone is free to start a school entertaining whatever niche you fancy. This solves the vast majority the pain of being in an informed minority - you can find educational institutions from pre-schools to universities where the administration focuses on excellence. For Math, Rutgers and Princeton are great Universities. For CS, I imagine Texas A&M would be good.

Any large university will give you enough of a selection of professors for you to be able to focus on the good ones.