Then you would have universities exclusively populated by people who are good at taking whatever type of standardized test you're administering.

Are you suggesting that having people capable of scoring well on exams of common necessities for college education, like mathematics and language skills, is not desirable? The SAT is not a trivia test: a student who can pass a test on math up through precalculus is going to get more value out of college than one who does not understand algebra and has to spend time in remediation. I would much prefer to have a university exclusively populated by people good at solving math problems and comprehending readings than by people selected for their "good vibes" and to fit a diversity quota.

You didn’t address the comment you’re responding to. You won’t have students who are good at math and language skills. You’ll have students who are good at taking math and language skills standardized tests. Being good at math and being good at taking a math standardized test is two different things. A student can excel at math but be terrible at taking tests for many reasons. You’ll have bright students falling through the cracks if this was the only consideration for admission.

How do you know that this hypothetical student “excels” at math if their performance can’t be measured? Students can and are already given reasonable accommodations. Sure tests may be imperfect but I am highly skeptical of any claim that math test performance and academic math performance are not highly correlated.

At the levels we’re talking about, the difference between “perfect score” and “still excellent but imperfect score” can be down to one or two errors on the test, basically noise. Given the number of applicants and the willingness to drill testing skills, going by scores would eliminate anyone who makes a minor error. The ability to perform perfectly in high-stress situations is a useful (and learnable) skill - like the ability to win quiz games, or perform classical music at concert level - but it is not the same thing as academic potential.

(As an aside: as an advisor of PhD students in computer science, I sometimes have to train students out of these habits. Research doesn’t happen if you’re used to getting the shallow answer more quickly, or if you’re afraid to be wrong.)

Yes, and that’s one of the reasons why tests are imperfect. That doesn’t mean that introducing any additional variables is warranted, especially given their associated noise. GPA is garbage, for example, which is why I ignore it on resumes. If the test has unacceptably high variance as you mentioned, the solution is to make the test longer not discount it.

I see no evidence that making tests longer will optimize for the variables that matter in academic performance (which are not 1-dimensional anyway) as opposed to simply optimizing for people who have poured resources into test preparation.

You seem to be coming from the perspective that testing must be the best metric, and then trying to develop a methodology based on that assumption. Purely as a thought exercise: it might be helpful to start from the assumption that testing isn’t an option, and think about what you might do instead.

Hard disagree about it being possible to be good at math but having trouble with something like SAT math.

Any student that can do any sort of undergrad level math like AP Calculus is going to find SAT math trivial in comparison. There's no way you can pass exams in the first but do poorly on SAT math, since all the stuff SAT math is testing on are prereqs for something like calculus.

I took the SAT decades ago, so the predictive nature of the SAT may have changed since — but I doubt that it has changed that much.

At that time, the SAT was good at predicting the likely performance of a prospective college student … for one term. That’s it. I got an excellent score on my SAT (not a perfect score, but I thought that it was a bullshit measure — which, like all standardized testing, it is — so I didn’t care). I did well in my first term, and then not so well after that (I ended up doing fine in the end, but it took longer to get through a degree program because I didn’t know what I wanted to be when I grew up; I think that Americans, by and large, put too much focus on going to university too early and we would be better served by an extra year of high school and/or a gap year).

One other thing I remember from that time period is that a better predictor of overall success in school was NOBITH, the Number Of Bathrooms In The House. That is, socioeconomic indicators were a better predictor than anything else, including nonsense standardized tests like the SAT. And given that socioeconomic indicators are a better indicator, that is a strong indication that there’s seriously wasted potential at all levels of eduction because we mistakenly equate family socioeconomics with success in general.

What would happen is you'd still get a result shaped by wealth and race, just laundered through a layer of standardized testing.

Which races are barred from wealth?

The post didn’t say barred. It said “shaped by.” Because of various historical factors, BIPOC are surely disproportionately “barred from” wealth.

Then we'd have heavy cramming, less fun, suicides and pressure like china south Korea or Japan. For better or worst.

That's how University of California admissions used to work for the most part. The formula was:

    Weighted GPA * 1000 + SAT Score + SAT II English + SAT II Math + max(remaining SAT II scores)

Then they ordered everyone by that number, and if say they were offering 5000 kids admission, the top 2500 would automatically get an offer letter. Then they would work their way down the list reading essays and adding bonuses for things like growing up in a rural area or an underfunded district with fewer AP classes.

They would then just keep admitting people until they hit 5000 offers.

Then they got accused of racism because that process ended up overweighting Whites and Asians compared to the applicant pool for all the top schools.

There is/was a lot of debate on why Whites and Asians did better, but when you dug in it turned out they tended to almost all be upper-middle class or wealthy. So basically it was wealth and systemic racism, masked by "standardized testing".

Also it should be noted that wealthy people had a better chance at a higher weighted GPA because their schools offered more AP courses.

This was going so well until the pivot in the last two paragraphs.

Is admitting the most capable students unfair? It is inherently discriminatory based on their knowledge and ability, which seems is what people want, but it also means that having more resources to be better equipped aka better nurture, which tends to positively influence individual's prospects, is suddenly something many are not okay with?

Obviously, the SAT tests are flawed one way or another. But then again, they don't care for students' superficial characteristics.

If not that, then implied statement is very reminiscent to slogans used in USSR or during cultural revolution in China.

> Is admitting the most capable students unfair?

Are they actually the most capable students, or did they just have a better chance of getting higher scores on the arbitrary metrics chosen? And if so, why did they have a higher chance?

There is a difference between better nurture and lack of opportunity. It's true, we shouldn't be correcting for lack of opportunity at the college admissions level, we should be doing it at the elementary level and younger.

But it's a lot easier to correct it at the college admissions level, which is a good choke point in one's life path, so for now, that is where the focus is.

Would different test-based or merit-based metrics radically change the result? After all, isn't the purpose of a holistic approach to college admissions basically to make the criterion not one of any measurement of quality merit, but of belonging to the right demographic mix that the college seeks to attain?

Further, why should colleges try to correct for lack of opportunity? Their job is to educate people, and not everyone is apt for higher education. Admitting a student with poor mathematical abilities into a rigorous engineering program is just setting that student up for failure, not recompensing him for opportunities lost or squandered earlier.

> Further, why should colleges try to correct for lack of opportunity?

Public schools should do it because they are part of the education system that needs correction.

Private schools arguably are still part of the public education system since they all get Federal dollars, and thus the same argument applies.

If they're getting tax dollars, that makes them a public good, and therefore they should be doing things that help all people, not just some people.

Additionally, if we are to expect that much of our political, economic, and technical elite are to pass through these institutions - and, further, that such institutions will trade explicitly on their ability to produce this elite - then there is an inherent public interest in the character of their operations. It's simply a matter of accountability.

You've accidental confused achievement and ability.

A student who goes to a poor school who offers zero AP classes will always achieve lower weighted GPAs than students who go to wealthy schools that offer many AP classes. No amount of individual achievement will overcome that mathematical disadvatnage.

Racism comes into when, because of racism, it just so happens that schools who are well funded and can give that advantage are overwhelmingly white, and schools who are are poorly funded and cannot give that advantage are overwhelmingly non-white.

Suggesting that a student who is "placed above his or her station" into an elite program is setup for failure is a favorite of a few SCOTUS Justices, but in fact is not supported by evidence. Graduation and other metrics of success are not strongly correlated to past attainment, primarily because most programs have already had to deal with achievement inequities, and the most successful programs already have mechanisms to even out unequal prior achievement. Virtually all programs that have elite programs already have a substantial apparatus dedicated to filling achievement gaps between incoming students.

The larger question of "what is the purpose of college" and "is it to educate people" is bigger than this thread, but shouldn't be overlooked. For schools that are publicly funded in whole or part, there should be a larger mission than sending young adults through an educational meat grinder. The public mission of public universities should absolutely have a social justice component.

There is absolutely no doubt that we could design an education system which, at an early age, divides and tracks students towards a successful and high level of attainment. By ruthlessly focusing resources on those children with the best chances of success, and minimizing resources expended on those with lower changes of success, the system could produce many multiples of positive outcomes than we do now, but at the cost of many more left with almost no attainment. Our present distribution of resources, in the US, is haphazardly assembled and produces a balance of outcomes, but is by no means optimized for any particular set of outcomes.

A student with a full family and access to tutors, with better mental health and social situation will be able to score significantly better.

However, the downside to "solving" this is significantly worse. By giving a headstart to students based on their superficial or socioeconomical characteristics creates perverse incentives in the system and disconnects it from a selection process.

Usually, countries that are not US solve this by providing free tutorships or lectures to aspiring students, scholarships and a fixed quota of free placements in the admission for top performing students (each country may have their own flavour of the policy, but the main thing they !fortunately! did not use to be based on race or gender).

It is usually much better, in my personal opinion, to provide the "help" to those who seek it rather than trying to equalize for outcome. Poor upbringing can and usually does damage and incapacitates individuals to an extent. But any movement among the lines of "not having issues or being better prepared means you have to do more work" explicitly discourages the optimal behaviour, incentivizing victim-hood and demoralizes individuals which did use the opportunities available to them or managed to create ones.

Again, and I cannot stress this enough, communism is bad.

I agree with you. This should not be solved at the college admissions level. Resources should be provided to younger students with potential to equalize there. But it's still better than doing nothing at all.

> Again, and I cannot stress this enough, communism is bad.

Again I agree with you. But I'm not sure how that's relevant. Changing the way we do college admissions isn't communism.

In fact, admitting students purely based on test scores is something that happens in communist countries.

> Are they actually the most capable students, or did they just have a better chance of getting higher scores on the arbitrary metrics chosen? And if so, why did they have a higher chance?

They are the most capable. You call them abitrary, without stating what makes them arbitrary. Until you prove they're arbitrary, your opinion isn't worth addressing.

Ironically present-day Chinese college admissions are arguably far more meritocratic than American college admissions. The main factor is performance on the Gaokao exam, an 8 hour test which dwarfs the SAT’s 3 hours.

I am too remote from this system to have strong opinions but I also heard all the princelings are doing great at those exams!

The problem with any standardized test is that it can be gamed with enough practice/resources that poor people are way less likely to be able to access. So it's not a cure-all, there are tradeoffs.

Standardized tests are the least game-able metric. In NYC poor Chinese immigrants manage to scrounge together $900 for an SAT prep class. It is much harder for them to figure out what some white college administrator wants to hear in a diversity essay. Or to get a job at some non-profit in China working on some social justice effort appealing to white college administrators.

Standardized testing isn’t perfectly meritocratic, but everything else is worse.

Absolutely. The cost of SAT prep courses are very small in comparison to the immeasurable cost of a lifetime of culture and experience that are necessary to impress an admissions board during an essay or interview. This is why it’s so absolutely disgusting to me to find that schools tended to rate Asian students so low on “personality scores”.

As a side note, I haven’t found any more detailed breakdown of the personality scoring results. Asia is a very diverse place and Asian ethnic groups in the US range from the very richest to some of the poorest.

Standardized tests are an imperfect metric. The fact that they’re imperfect doesn’t make all metrics equivalent.

For example, Harvard could simply auction off their seats. That’s transparent at least, if not exactly fair.

But it is still the fairest option we have. Any other metric is usually much more gameable. Be it sports, essay writing, extra curriculars, volunteering?!?!? and so on.

Or maybe alternative is truly random or negative wealth scoring. That is only the poorest students get entry and everyone else will have to figure out something else.

> Is admitting the most capable students unfair?

Grades and standardized test scores might be one of the least interesting and almost definitely one of the less effective ways to classify the “most capable” students.

If I’m optimizing for students who do cool stuff, force marching them into relatively banal optimization tasks is counterproductive.

Strongly agree with the sentiment. In fact, I alluded to this in the parent response by mentioning that SAT and most other testing systems will be always flawed. The question itself was posed with testing criteria issues notwithstanding.

With that said, I think modern education system can no longer scale and has outlived its usefulness. The future is with LLM-driven tutors which can somewhat alleviate the scaling issues inherent to "apprenticeship". Personally, as an ADHD individual it was always drastically easier to study under a person who gives you dedicated attention that for any of my peers. By a wide margin. And yet, studying for local SAT equivalent was an insurmountable task involving acquiring a lot of knowledge and skills that were useless outside of passing the test.

The sad thing is that students from poorer and racially segregated areas are paying for 18 years of falling behind by the time it comes to college. When some high schools have barely literate graduates betrayed by adverse conditions their whole life, it's hardly shocking that it results in college admission readiness being grossly skewed. Not that I want to oversimplify.

And why not fix this issue? There is multiple options. You could build something like large prisons somewhere. Ship all the kids there from age 6 to 18. And then shuffle them randomly around every 6 months. Thus ensuring everyone gets exactly same experience and education. Equity for everyone!

This is a delibrately ridiculous suggestion.

It would be more constructive to improve teacher education, wages, and standards to create a more productive and better resourced education system.

You could look to countries such as Finland et al for a model rather than drawing upon satirical dystopian ankle deep in thought Harrison Bergeron scenarios.

I'd start with making property tax go to a central pool before getting distributed back to schools instead of letting the wealth stay concentrated in wealthy districts that have often been formed mostly to avoid sharing revenues with less affluent areas.

Would be interesting to see what would happen if they applied that formula per each high school, not per graduating body, with each high school getting some guaranteed college slots. Maybe poorer schools would get more attention this way.

That would basally be admitting the valedictorian and salutatorian from each graduating class.

My understanding is that Brown basically does that. This would sort of back that up:

https://collegeshortcuts.com/blog/brown-acceptance-rate

It would be a lot worse. Harvard could fill its classes 4x over with students with perfect SATs and GPAs. They would miss out on the ones who published ground breaking biology papers, wrote code for important open source projects, etc.

It would also put a ton of pressure on high school students to get perfect scores, creating a ton more stress than they already have. And at the expense of much more valuable things they could be doing with their time.

The current system has issues, but there are worse ways to do things.

This suggests to me that the SAT isn’t hard enough then. That many students should not be able to get perfect scores.

The number of high school or undergrads who wrote groundbreaking biology papers is vanishingly small, but you do raise a good point. There’s a chicken and egg problem. Because schools aren’t just weighting test scores (because the score is flawed) there is incentive to work on other, more subjective things.

> This suggests to me that the SAT isn’t hard enough then

That’s tricky. In a massive country with many incredible students, it seems to me that it’s going to be possible for the top .1% of students to do as well as is possible on any reasonable exam.

Do you disagree?

Then you have tiered exams. e.g. if you score >1500 on the SAT, you can take the harder SAT-2 which would be used for admissions to top schools.

Diminishing returns, really. As you continually test the margins will get smaller and smaller and the outcomes won't change much. Someone scoring >1500 on the SAT will likely just as well graduate from ivy league as someone who scored 1550.

Someone with a 1300 would do fine. But only because thy aren't as difficult as top schools should be because, as we see from their admissions policies, they are status and credentialing gatekeepers to elite status, not really top educational institutions.

Right - so does it really matter?

If wealth is the ultimate playground for everything (capitalism?), then obviously the elite institutions are playing the right game.

Yes it is small, but you want to incentivize that, because someone smart enough will make it happen. Or you can be smart enough to do both.

You are describing the French elitist educational system until maybe 15y ago. This counterintuitively resulted in very little diversity.

There might be natural reasons for that but there were also clearly some perverse effects: because those exams became effectively competitive exams (fixed number of seats available), they require intense preparation in preparatory schools which became the educational equivalent of a 2-3y navy seal bootcamp (it's not enough to have good grades, you need the best grades). Teachers in poor area high schools were often unfamiliar and intimidated by this system (they didn't go through it themselves), and were actively discouraging good students to take that path. Students not having educated parents who knew the system were not pushed to work really hard to get into, and survive that system either. And you ended up with 80%-90% of the kids in those colleges coming from families from similar colleges. Money wouldn't be a major factor, the top colleges and preparatory schools are free.

Now the natural ratio shouldn't be 3-5% (elite college educated parent as % of general population), probably around 50-60%, certainly not 80-90%.

For the past 15y, France has opted to break the thermometer instead, ie to bypass those exams for the majority of the recruitment, and to select students on file which grants them full latitude for subjectivity. I suspect they also nationalised the selection process of those preparatory schools (not super familiar with the latest developments). I think the main consequence is that those elite colleges will quickly lose their prestige, and it might actually be the unstated intention, i.e. converge to a german like education system where the name and prestige of the university matters little vs the profile of the student.

It might be a good thing or perhaps it will just ruin the french educational system, it's hard to tell.

I think it would work better with a slight twist: set a reasonable score floor, run a lottery for everyone above it. Instead of giving the world to people who(se parents have paid to) optimize their studies to a particular test, everyone who proves a certain degree of capability has a chance. Students simply do their best, and fate and the RNG takes care of the rest. If it becomes a problem, that we're underserving too many bright and promising youth because too few are getting into Ivies under this regime... maybe create more Ivy-likes (Stanford says hi).

And for anyone who thinks it's unfair: not anymore so than the parent lottery. And I think institutional acknowledgment of the role pure dumb luck plays in our lives would go a long way towards dispelling our inclination to actively punish people for acts of the cosmos and/or dysfunctional policy.