I think it's wrong to believe the claims that Yale, Harvard, etc. have actually become meritocracies.

They absolutely are not. For undergrad, these ivy league schools are much more forgiving of grades from elite high schools.

"Well rounded" or "holistic" admissions also strongly favor upper and upper middle class activities and talents.

Admissions are gamed to such a large degree that it's all a game that is played out year in advance. And the middle class and poor students don't even know it's being played.