I've made this proposal several times: judge an applicant by his/her delta relative to their cohort. For example (highly simplified) a kid from a wealthy family in a nice neighbourhood statistically scores 70% on average, if he scores 80 that's a +10. A kid from a poor single-parent household going to a dilapidated school where scores are 40% on average, if he scores 70 that's a +30.

Very rough example but the idea is this. Philosophically/ethically, it seems better. But practically as well: you want to admit the students most likely to excel if given the high quality resources at your university. Among a kid with all the comforts and private tutoring and all that, and a kid who rose way above his shitty lot in life, wouldn't the latter fit that description better?