More importantly your two positions are not excludable. It is possible that the elite of the past were wrong and some segment of the population is incapable of being raised any higher. I personally doubt this is the case, but it has not been ruled out.
I also suspect that we are a long way off that, hell who knows what the future looks like, we could end up with neural implants that allow everyone to boost their intelligence/recall.
I'm not stupid but I know there is an upper bound on my intelligence that is lower than many other peoples, not everyone can be Einstein or Hawking.
The more interesting problem is how do we optimise for people reaching their maximum potential or if that is even necessary.
(We, as a species. The particular education system I went through scores much better than the US system in every benchmark, but it's still piss poor at actually imparting skills on pupils, instead of rewarding them for accidentally ending up with them for reasons nobody really understands. I don't think any system has properly figured that out yet.)
But as it turns out, almost all of those peasants were capable of learning to read, doing algebra, engaging in abstract reasoning, labor coordination, and operating some fairly complex machinery. Many of them were even capable of becoming software engineers. We know this, because pretty much all of us came from them, and if we can do it, they could've done it, given the proper environment. Someone advancing your argument 1000, 100, or even 50 years ago would've wrong, what makes your argument different?
Of course we shouldn't expect equal outcomes from all people, but to suggest that we've now reached such a level of access and fairness in society that everyone still at the bottom can't possibly be elevated any further seems... unintelligent.