Then you gotta make it affordable for them.

You can still have elite schools for the absolute best (though I would question that certain self-styled 'elite schools' are really principally selecting for that and not e.g. wealth or family connections), but that doesn't mean that you cannot raise the bar on public and affordable education (including traditional schools and universities but also via other means, e.g. self-teaching via online resources). These are two competing goals and often people only focus on the one to the exclusion of the other, but they are both important.