It's possible that they will become less biased in their admissions, but it's also possible they'll become more biased.
The University of California has also announced that they won't be considering SAT/ACT scores for admission. California forbids affirmative action in their state schools and while the SAT/ACT aren't objective tests, people often treat them as such and it provides a number for people to compare (whereas you can't really compare "really good at guitar" with "volunteers planting trees"). It's led to Berkeley being 2% Black and UCLA being 3% Black while Stanford is 7% Black and USC is 5% Black. Also notable, Berkeley is 24% White and UCLA is 26% White while Stanford is 32% White and USC is 37% White; Berkeley/UCLA are 36%/28% Asian vs 23%/21% for Stanford/USC. The University of California system is certainly having its admissions decisions impacted by California laws against affirmative action in their public universities.
I don't expect Harvard's admissions makeup to change significantly. I just expect it to be harder for third-parties (who might want to sue) to compare data. If you're qualitatively comparing applications that take a lot of time to review, it makes a lawsuit very difficult. If you're able to compare a numeric score, it makes it less time-consuming (even if the numeric score is biased/inaccurate, people often treat it as unbiased/accurate). It's easier to argue qualitative differences in judgement too and means that you have to start challenging things individually rather than as a class (which basically makes it impossible).
With the University of California, it will be interesting to see if they start coming closer to peer institutions in their student body.
Still, I'm a fan of standardized tests. If there's a problem, we should try to improve the tests rather than doing away with them.
But I think the bigger danger is that after a while, practicing for standardized tests starts monopolizing the curriculum at secondary schools, and teachers start being evaluated by their students' test scores to the exclusion of all else. Even 20 years ago, I remember getting herded onto a bus to some community college to take an extra "practice state proficiency test" whose score didn't count for anything at all.
Even if universities have lousy motivations, de-emphasizing that kind of thing probably makes high-schoolers' lives a little better.
One could make a test called 'big brain test' and sell it to schools to say it measures big brains, and it might be only somewhat correlated but still get widespread adoption because it fulfills a need schools have to have some sort of quantitative sorting of students, but it still wouldn't necessarily be a measure of big brains. This is sort of what the history of the SAT is.
Here's a comment from a different branch of this thread:
> The findings in general is that much of the disparity between minorities and whites goes away for harder analogies that are more technical or scientific in nature, in fact contrary to what I expected, the recommendation of the study is to simply eliminate the bottom third of verbal questions (ranked by difficulty) involving analogies or sentence completion, as those are the questions that are most likely to involve cultural bias.
Secondly, that doesn't address differential prediction abilities. It addresses a concerning aspect of the verbal test, but is somewhat irrelevant when your goal is to predict college success. I'm not aware of differences on that metric.
That's good to hear. It's hard to know what research is still fresh, unless one is employed to stay current in a field.
so imo getting rid of standardized tests does get rid of a subjective rating of an applicant which is good in theory, but when you leave in all of the other subjective ratings that are much more difficult for disadvantaged students to do well on it who is it actually benefiting?
That doesn’t make a timed foot race a subjective measure, IMO.
I mean, it's okay to use the best imperfect metric you have available; so acknowledging "the metric is not fully correlated and depends on other factors" does not mean anything, the only argument that has weight is "hey, this other metric has better correlation with The True Thing and less bias, use that instead"; if the remaining alternative metrics are even less objective and more biased than SAT/ACT tests (which IMHO is the case), then removing SAT tests won't and can't improve things, despite these tests being flawed.
to be clear, i still think the sat is better than everything else because preparation is much more legible and accessible compared to all of the other ways to stand out for college applications.
Pointing out shortcomings of standardized tests is one thing. Advocating for eliminating what seems to be our most equal/standardized/objective/whatever measurement without a replacement seems like a bad idea to me.
https://slatestarcodex.com/2015/06/06/against-tulip-subsidie...
In my not-so-humble opinion, however, the best way to prep for the verbal SAT is to read books. Novels are fine.
For the math, prep by taking all the math classes your school offers, and make an effort to learn all the material.
Do that, and you won't need prep classes, and you'll also be well prepared for college.
Which begs the question, who exactly do you admit to your school, and why?
They can still take the test and get a high score.
> How are the SAT/ACT not objective tests? Are they not objective measures of how you perform on the SAT/ACT?
Most of the studies on this show that all of the cultural references in the reading passages and math word problems were things that wealthy people would have experienced in real life, but poor people would not, so it gave an advantage to wealthy kids. Also a lot of the vocab were words that wealthy kids had heard at home but not poor kids.
So while it was supposed to be a test of intelligence, it is really a test of how wealthy you were growing up.
Edit: Here is the main study from 2003 from Harvard: https://meridian.allenpress.com/her/article-abstract/73/1/1/...
Here is a writeup of someone who replicated it with more recent test scores: https://www.insidehighered.com/news/2010/06/21/new-evidence-...
Poverty makes things harder, no doubt. But American parenting is just bad and American students don't try very hard. My wife comes from a lower-income white family (multi-generational lack of economic mobility) and the lack of emphasis on eduction or discipline at home is glaring. Some people, like my wife, are just naturally very self-driven and make it out. But a lot of potential gets wasted because kids are allowed to just faff around instead of being rigidly guided by parents (or, as is more commonly the case, by the one parent).
The findings are that much of the disparity between minorities and whites goes away for harder analogies that are more technical or scientific in nature, in fact contrary to what I expected, the recommendation of the study is to simply eliminate the bottom third of verbal questions (ranked by difficulty) involving analogies or sentence completion, as those are the questions that are most likely to involve cultural bias.
If my understanding is correct (and I'm sure I have some subtle details wrong), the idea is to let students take the SAT, then go over the verbal/analogy/sentence completion portion of the SAT and order each question by the proportion of white students who answered it correctly, and then eliminate the easiest of those questions from the final score. The most difficult questions are still part of the test, but apparently most of the cultural bias is among the easiest questions to answer that minorities get tripped up on.
If your hypothesis were true, that it's mostly a matter of bad parenting, it's kind of hard to see why bad parenting would make minorities struggle to answer the easiest questions for white people, but not also struggle to answer the hardest questions as well.
[1] https://www.insidehighered.com/news/2010/06/21/new-evidence-...
This is all you needed to say.
The problem is that the system is designed to identify the most effective parents, not the most talented children. If you look at college admission as a function of personal merit, then it's unfair to judge a child based on their parents' ability to raise them. If you look at it as a system to identify those who will benefit society most by being educated, then you're going to get a lot of false positives that have parents from higher socio-economic classes.
Providing a test that produces an empirical result is just making it very easy for parents to game the system. It's a single, stationary target that they can fire money at. If you take away the test you're still selecting for the best parenting (all of life is). But it lets universities put more emphasis on grades and personal stories, areas where students whose parents have never heard of the SAT can still excel.
If you can use money to game the test then it's culturally loaded toward people who have enough money to game the test. A broader admissions criteria doesn't have this specific flaw
And you’re missing the point re: income, which is that all poor kids aren’t similarly situated. Asian kids raised in the bottom 20% of the income scale have a 25% chance of ending up in the top 20% as adults. For white kids it’s only 11%. Socioeconomic status is a factor but it’s not dispositive or insurmountable.
As to grades—you think it’s easier for a bright but poor kid to do their homework in time for 12 years than to study for one test? As to “personal stories”—that’s just a vector for manipulation by the whites people who run the admissions system. It’s remarkable to me to read anecdotes from admissions counselors, who say stuff like that they are looking for weird and non-conforming kids: https://www.cnbc.com/2021/10/22/former-stanford-admissions-o.... Talk about cultural loading!
I never said that someone's income quantile was dispositive. If my parents never took the SAT and have no idea what it is, I'm at a huge disadvantage to do well on the test by any measure. Regardless of their income. Again, the SAT isn't testing for academic aptitude but for the cultural knowledge of parents and peers.
As a wealthy parent I know exactly how to throw money at prepping for one test: hire expensive tutors and purchase practice tests. Gaming years of GPA or coming up with a compelling story is more complicated and difficult to solve with the application of financial resources.
A system with vague and subjective criteria is subject to bias, for sure. They are also more difficult to manipulate directly with wealth (other than bribes). We have to decide what our priorities are there as a society.
And that's just the reading comprehension. The idea that the math word problems involve cultural bias that would affect performance is utterly ridiculous. "Most studies"... there is no chance that any quality research has been done to support something so laughable.
I will look at any citations you may have, but I am 100% confident that you have been duped.
You can definitely cram for the grammar / math portions, but the reading comprehension sort of just requires a lot of reading experience. There isn't a rote set of things you can memorize to get a high score, which could arguably be seen as good or bad.
Maybe more practically, we should consider: Do lower income students read less on average? And if so, is the difference greater than their ability to gain access to tutoring services / prep books which can be used for the crammable portions of the exam?
That 2003 study was done at Harvard.
Here is a writeup of someone who replicated it with more recent test scores: https://www.insidehighered.com/news/2010/06/21/new-evidence-...
Well, I'll be damned. Huh.
> The 2003 study and this year's found no DIF issues in the mathematics section.
The earlier claim about math word problems being culturally biased seems unsupported by the citations.
But will the claim be withdrawn? Hmm, let's find out...
Runner is to Marathon as XXXXXXXX is to Regatta
If you grew up on the coast in a wealthy family, you have a much higher chance of knowing what a Regatta is and that it involves boat races.
There are no questions like that any more. You really think the people designing the highest-profile standardized test in the country are oblivious to this kind of problem these days? There's a ton of cultural and racial bias screening and filtering that goes into producing these questions now, both manual and statistical. You can look at some sample questions yourself.
It would be hard to deny that generally sailboat racing is done by wealthy people.
The yearly regatta has always been like the Olympics of Newfoundland
Incredible educational resources are available online for free and just a click away on that supercomputer in everyone's pocket.
There's no reason to be restricted to your parents' vocabulary.
Runner is to Marathon as XXXXXXXX is to Regatta
If you grew up on the coast in a wealthy family, you have a much higher chance of knowing what a Regatta is and that it involves boat races.
But if you read the papers cited in the GGGP comment, those hard problems aren't the problems that show the racial gap. The hardest problems show the least gap. Which doesn't fit the pat little sailboat jargon theory at all. However it does fit a common pattern, also seen in longevity statistics, where the most exceptional are more similar to one another than the average are.
Wait, can you elaborate?
As people get older, the mortality rate gaps between different groups get smaller on a relative basis. Like, the ratio between rich and poor 50 year olds' risk of death over the next year might be 3x or more, but as they age, they become more similar, and by 90 the ratio may be more like 1.3x, and after 100 it becomes indistinguishable from 1 (in part because of thin data).
The theory is that the genetics and environmental conditions needed to survive to an old age are fairly specific, so once you're down to those people, they have less variation from one another. The health risk factors that distinguish income and racial groups may mostly impact younger age mortality, and the people that get past those younger ages may have avoided or resisted those risk factors, regardless of race or income.
Similarly here, students who "survive" to do well on hard SAT questions may be fundamentally very similar to one another (sharing a high g-factor), and this may greatly reduce socioeconomic and racial gaps that are seen on the easier questions.
Does this mean the easier questions are biased? Not necessarily, I would argue. The concepts tested in the easy-medium questions are most likely to be intellectually foundational -- you are far, FAR more likely to get obscure rich people jargon in the hard questions than the easy-medium ones. We want people to have the basic intellectual exposure that enables them to do well on those easier questions, and we should reward it.
The problem is not the test, the problem is racially biased failure to educate in the lead-up to the test. This explains the pattern perfectly: the strongest students power through regardless of background, while the weaker are more sensitive to resources they were given. So you see the biggest gaps on easy-medium questions where basic educational background matters most to getting them right, rather than raw talent.
Have you explored broader implications of your idea? For example people who survived tribulations of a political career may end up close to each other despite their initial difference in political opinions, and maybe even across different countries. This doesn’t sound quite as convincing, but I’m kind of grasping in the dark trying to find some other examples.
I don't know of any examples outside of health but health is a big enough topic and closely related enough to education that I think it's likely to have some application.
Certain psychometric studies also support higher sensitivity to environment in the lower economic strata. Not exactly the same but you see the connection.
The studies cited by jedberg are very interesting, I just don't think they show what he thinks they do.
A better alternative to trashing the SAT is to provide an estimated adversity score, as ETS recently began to do, so schools have a sense of what a student may have had to overcome to reach their level of achievement.
Still there is something in the longevity example that’s evading the trivial explanation.
It's also a little unclear whether we should consider the % difference convergence remarkable, given that mortality at older ages is so much higher and the absolute difference may not show much convergence.
MENSA accepts it for admission to their high IQ club.
Depends on how much money is meaningful to you. Time has its own costs, especially since an older teenager can work as an alternative.
You take the test twice, once with a blindfold on, and once without a blindfold. As an objective measure of your intelligence, I expect you to get the same results on both tests
You are considering whether it's objective only by that the reader at the end sees the final score, but there's many spots in between an empty sheet of paper and that score where somebody's subjective experience comes into play
We can argue about whether or not the stuff on the test is important. We can discuss the various environmental, cultural, and socioeconomic factors that go into the kids taking it and how that impacts scores statistically. We can even talk about the various medical and neurological factors that go into test taking generally and standardized test taking specifically that can affect some people more than others. I'll even concede that the SAT/ACT isn't a test of your intelligence so much as it is a test of how well you did on the SAT/ACT, just like putting things on a scale may at best be a measure of its weight relative to other things on that specific scale.
But at the end of the day, more right answers === a higher score, and I'm not sure how you get any more objective than that when it comes to something as fluid as academic ability.
Getting rid of one of these because it's easier if you have money (when all of these things are easier if you have money, because life is easier if you have money) is just setting the stage for more explicit bias.
By comparison, Asian and Pacific Islander populations combined only make up 15% of the California population (1). I don't suppose the interaction of affirmative action policy and cultural differences might have inadvertently advantaged a couple of Asian subpopulations that are now pissed that the basis of their advantage is being taken away?
(1) https://www.ppic.org/publication/californias-population/
I was one of those students who got perfectly scores on all my tests but couldn’t possibly be bothered to do homework in high school, so my grades were less than perfect. If it weren’t for my high ACT score, I’m pretty confident I wouldn’t have been accepted into any university—especially as someone from a lower middle class family.
I’m not sure if eliminating standardized testing is a misguided effort to introduce fairness or a wisely guided effort to only let in the types they want through a more opaque process.
I was a mere peasant at a SUNY school, but I tutored kids in the EOP program. These kids were mostly from awful high schools and had aptitude, but lacked a lot of basic high school skills.
Just like my friends from “normal” educational backgrounds, the kids who worked did fine. Most were boxed out of engineering but I know of a few alumni who are attorneys or have other professional gigs.
If you back basic high school skills then you're not only probably not suited to Engineering, you're also probably not going to cut it.
It takes 'Aptitude' plus at very least skills plus even more.
Quiet part aloud right here.
What you're saying is that if you haven't been raised in a certain way (in an environment that prioritizes formal schooling) then you're not fit to be a professional (engineer in this case) no matter how smart/talented. There's a word for this: classicism.
But I'll tell you something. As someone who grew up exactly like that but is now getting a PhD in CS at a top school and who has excelled in several FAANG internships: you can teach/learn high school skills. You can't teach/learn aptitude though. So if you think it's some kind just/effective/correct that these kids are kept out (just because they don't have good essay writing skills or something) well I have to wonder if you're just someone trying to keep the labor pool small to inflate your own salary (or for other reasons...)
You can't teach 'aptitude' but I think you can encourage those things i.e. reading, curiosity, exposure.
You're right though that rich kids probably have a huge advantage in prep for Eng in terms of the former, but whether or not you call it 'classist' is besides the point: teaching your kids to show up to class and pay attention, do homework etc. is not something that costs money (of course it can help).
"So if you think it's some kind just/effective/correct that these kids are kept out (just because they don't have good essay writing skills or something) well I have to wonder if you're just someone trying to keep the labor pool small to inflate your own salary (or for other reasons...)"
For a PhD you're having an odd bit difficulty with leaps of unsubstantiated assumptions, because there's definitely nothing in my statement that would remotely hint about 'What I'm trying to do' about anything. I mean seriously, "I'm trying to keep the labour pool small?" What?
of course it does. it costs money in the form of time - you have to be not working at home to have the opportunity to teach these things. it costs money in the form of energy - if you've never argued with a young child about the relative merits of homework and television then you have no idea how much it costs.
>because there's definitely nothing in my statement that would remotely hint about 'What I'm trying to do' about anything. I mean seriously, "I'm trying to keep the labour pool small?" What?
really? then what is the purpose of this part of your response?
>then you're not only probably not suited to Engineering
what do the words "not suited" mean to you?
There is not much to interpret from that statement.
'Basic High School Skills' would be general subject matter competence, basic diligence and conscientiousness with respect to attendance, participation, learning, homework, socialization, organization.
Engineering is fairly advanced, it requires an even higher degree of general competence than most Uni subjects, and even those need a level of competency only found in the upper tranches (say top 1/3) of students in high school.
If you're not 'Generally Not Good At High School' then you are not going to make it through Engineering.
I don't think there's anything controversial here.
begs the question.
>and even those need a level of competency only found in the upper tranches (say top 1/3) of students in high school.
yet not a single one of these things has the slightest to do with technical aptitude
>'Basic High School Skills' would be general subject matter competence, basic diligence and conscientiousness with respect to attendance, participation, learning, homework, socialization, organization.
i'll repeat myself: you can teach each of these things to a kid that is good at math and physics but you cannot teach a kid that has perfect attendance, diligence, etc etc etc how to be good at math and physics.
>If you're not 'Generally Not Good At High School' then you are not going to make it through Engineering.
<shrug> i made it through a physics+math BS, and I'm well on my way to finishing the PhD (as in aced my classes and my quals) and i graduated high school with a 2.2GPA and 40 absences senior year. so not just bottom 2/3 but probably close to last. so along which axis do you think you're wrong? either a technical degree doesn't require the kind of "diligence" you think it does (i'd argue it does not) or that diligence can be learned fairly easily (i'd argue that too).
yet not a single one of these things has the slightest to do with technical aptitude"
?? Aptitude is definitely correlated with academic performance. There is no debate there.
I'm sure on your journey you've taken enough stats to grasp that your personal anecdotal experience doesn't count for that much? I mean, being last place in school, absent all the time ... would it be reasonable for you, the Uni, or anyone to believe that you were 'well suited' to Engineering, or at least more suited than those with good grades, GPA yada yada? It's great you did well, but you must agree that wasn't likely.
In the aggregate, both GPA and SAT are highly correlated with Academic performance in University, so you're arguing against the wind here. [1]
(Do I really even need to provide a data point on this?)
And you must know that being 'Last Place' in High School would preclude most kids from even being accepted to Eng. programs, let alone Uni.
Students who do poorly in High School generally won't succeed in Eng. programs - let alone be accepted in the first place.
Everyone knows this, and it's why they use Grades and SAT as a primary means of admission.
And FYI raw aptitude can't be taught but all sorts of other things can i.e. having basic mathematical literacy and just 'keeping up' from grades 1-9, means that kids have the confidence and opportunity to participate in 'STEM' things which they would be blocked from doing otherwise. Peers, Mentors, points of Inspiration also give kids the extra energy and ethos to work through the issues they might not otherwise care about, in effect, there's a lot of 'passion' hidden inside 'aptitude'.
[1] https://www.forbes.com/sites/prestoncooper2/2018/06/11/what-...
You're literally commenting on a thread about how the SAT is being removed from criteria and your whole point is that the intangibles (you used a bunch of ambiguous words like diligence and etc and I explicitly pointed that out). So have you lost the thread of the conversation?
>Students who do poorly in High School generally won't succeed in Eng. programs - let alone be accepted in the first place.
You really need to look up and understand the begging the question fallacy that I alluded to
https://en.m.wikipedia.org/wiki/Begging_the_question
>And FYI raw aptitude can't be taught but all sorts of other things can i.e. having basic mathematical literacy and just 'keeping up' from grades 1-9, means that kids have the confidence and opportunity to participate in 'STEM' things which they would be blocked from doing otherwise.
I do not understand what you're saying. On the one hand not being diligent and having good attendance should preclude students but lack of "confidence" shouldn't?
The successful students were taking and succeeding at Calc 1 in their second year.
- applicant A: average grades, average SAT, no sports
- applicant B: great grades, no SAT, great sports
- applicant C: great grades, great SAT, great sports
then B wasn't even considered. Now, B will probably get in over A.
That's because the admissions process is not the line-by-line ranking exercise you're implying.
In some cases an SAT may help to complete the picture of an applicant. In other cases, other attribute permutations are enough. Maybe #1 was close and the SAT sealed it. But, it's not necessarily the case that applicant #2 was hurt by not submitting their SAT (or would have fared any better by submitting even a perfect SAT score).
For tests, simply paying attention and listening to what the teacher said was enough to do well.
For many people, homework is completely useless and many US schools absolutely love overwhelming students with it. There are also kids who need to work jobs after work to support their family--busy work assignments are just punishment for them, and countless people I know suffered because of that. The ACT/SAT was a way for them to elevate themselves.
just the way they refer to it is disturbing, bad enough favoring legacies but "lineage" makes it sound like they really think these people are somehow superior based on their bloodline. The new nobility, how dare these serfs with good test scores defile our institutions!
It's worth noting that the term "meritocracy" was actually coined by somebody criticizing the serfs defiling sacred institutions.
Basically meritocracy can turn into classism, and due to accumulated advantage it can entrench. Everyone has a chance in theory, but in practice advantage compounds.
There's also the simple human welfare problem that vicious never-ending competition every minute of your life is exhausting & demoralizing.
In practice this doesn't happen - some children have massive advantages compared to others. To then take all of these children and one day try to use an objective measure to decide who gets access to yet more advantages is obviously going to massively favor the non-meritocratic-to-that-point advantaged children.
So yes, a true meritocracy may be completely fine, but in the real world "meritocracy" is. anything but.
Without a common language communication becomes impossible.
The normal definition of the word is that those who are best at a task should perform it. The most competent applicant should be hired for a job. If a billionaire's kid happen to be best, because of all the private tutoring they got, that's not really relevant for a meritocratic decision.
I mean, if some job requires being good at math, I agree math ability should be considered. It's just that I don't think it should be high status or prestigious.
Harvard has always been a social indicator that is also decent educationally in some areas (but not, AFAIK, superlative in any undergraduate field.)
You guessed it, poor whites and asians. The kids who couldn't afford extra curricular but who did great on the tests. Who is it going to help the most? Children of wealthy East-African and Caribbean immigrants.
This focus on extra-curricular also makes it really easy to sneak-in affirmative action (despite voters and the general public repeatedly saying no). Because, let's be honest, you can't objectively compare these things to one another.
> lower scores amongst legacy and athlete students
I'll give a pass to athletes.
It's no small feat to get an athletic scholarship, and for many sports there's absolutely no future beyond college (no pro league or circuit like football or tennis).
The unfortunate thing may be that we've conflated college admissions with this notion of "merit", rather than what you can get out of college itself.
Of course, I'm sure the athletics department does apply other forms of pressure to various parts of Harvard, making it easier for athletes to get admitted and pass their classes etc. But any scholarships they award are (corruption aside) at least theoretically based around financial need.
(Since it's hard to tell in text: no I'm not being sarcastic.)
I agree with most of your comment, but I think you're using "harder" and "rarer" interchangeably here.
One would guess that receiving a scholarship for elite athleticism or superior intellect are both quite difficult, and I have zero basis of comparison for which one is harder. I would not assume that athletic scholarships being rarer reflects on their difficulty. Rather, it's most likely a reflection on what the main function of a university is, as a place of learning. They admit more students to study than to play sports.
I’m guessing it also helps eliminate the “blue collar asian” applicants more easily. It removes one of their shining accomplishments.
From a 2018 article from politico:
https://www.politico.com/magazine/story/2018/10/21/harvard-a...
“But the data collected by the Education Department contained some explosive information. It showed the athletes and so-called legacies who were actually accepted had lower SAT scores than the rest of the class and were also deemed less attractive candidates by the admissions officers conducting Harvard’s process.
Some of the comments those officers wrote on the application folders of admitted legacies strongly suggested something more than a tiebreaker was at work. “Lineage is main thing,” one reader wrote. “Double lineage, but lots of problems … no balance,” the notes on another successful application said. “Lots of lineage here … Hard to explain a NO,” yet another said. “Classical case that would be hard to explain to DAD.”