This is pissing on people's leg and telling them its raining. There's no "cultural loading" in the SATs. English is my second language and I didn't even know Chicago was a big city until my mid-20s, but reading a lot and practicing vocabulary words is easy and cheap. New York's Stuyvesant high school has an average SAT score of around 1500, and half of students are eligible for free/reduced price lunch: https://www.nydailynews.com/opinion/stuyvesant-serves-needy-...

Poverty makes things harder, no doubt. But American parenting is just bad and American students don't try very hard. My wife comes from a lower-income white family (multi-generational lack of economic mobility) and the lack of emphasis on eduction or discipline at home is glaring. Some people, like my wife, are just naturally very self-driven and make it out. But a lot of potential gets wasted because kids are allowed to just faff around instead of being rigidly guided by parents (or, as is more commonly the case, by the one parent).

So then what do you make of the studies that show that SATs are culturally biased, like the one OP posted [1]. I decided to go ahead and buy it for the 10 bucks and review it. It's a surprisingly technical study that seems fairly robust and it has been replicated several times (original study in 2003 and replicated in 2010).

The findings are that much of the disparity between minorities and whites goes away for harder analogies that are more technical or scientific in nature, in fact contrary to what I expected, the recommendation of the study is to simply eliminate the bottom third of verbal questions (ranked by difficulty) involving analogies or sentence completion, as those are the questions that are most likely to involve cultural bias.

If my understanding is correct (and I'm sure I have some subtle details wrong), the idea is to let students take the SAT, then go over the verbal/analogy/sentence completion portion of the SAT and order each question by the proportion of white students who answered it correctly, and then eliminate the easiest of those questions from the final score. The most difficult questions are still part of the test, but apparently most of the cultural bias is among the easiest questions to answer that minorities get tripped up on.

If your hypothesis were true, that it's mostly a matter of bad parenting, it's kind of hard to see why bad parenting would make minorities struggle to answer the easiest questions for white people, but not also struggle to answer the hardest questions as well.

[1] https://www.insidehighered.com/news/2010/06/21/new-evidence-...

The racial gap is almost exactly as big on the SAT Math as it is on Verbal. Do you think SAT Math is also culturally biased? If so, do you think it is exactly as biased as Verbal? That is, why is the Math gap not, say, half the Verbal one, if the cause for disparities is cultural bias?

Why would there be a Black-white gap (two groups that come from the same American culture) but not an Asian-white gap (two groups that come from different cultures)?

Black and white families are not in fact part of the "same American culture". And, of course, it's not the case that Asians uniformly outperform white subjects on the SAT; "Asian" comprises many different cultures, and not all of them are empirically advantaged in America.

> "Poverty makes things harder, no doubt."

This is all you needed to say.

The problem is that the system is designed to identify the most effective parents, not the most talented children. If you look at college admission as a function of personal merit, then it's unfair to judge a child based on their parents' ability to raise them. If you look at it as a system to identify those who will benefit society most by being educated, then you're going to get a lot of false positives that have parents from higher socio-economic classes.

Providing a test that produces an empirical result is just making it very easy for parents to game the system. It's a single, stationary target that they can fire money at. If you take away the test you're still selecting for the best parenting (all of life is). But it lets universities put more emphasis on grades and personal stories, areas where students whose parents have never heard of the SAT can still excel.

If you can use money to game the test then it's culturally loaded toward people who have enough money to game the test. A broader admissions criteria doesn't have this specific flaw

There is no such thing as “personal merit” independent of parenting. Kids are a product of their genetics and upbringing.

And you’re missing the point re: income, which is that all poor kids aren’t similarly situated. Asian kids raised in the bottom 20% of the income scale have a 25% chance of ending up in the top 20% as adults. For white kids it’s only 11%. Socioeconomic status is a factor but it’s not dispositive or insurmountable.

As to grades—you think it’s easier for a bright but poor kid to do their homework in time for 12 years than to study for one test? As to “personal stories”—that’s just a vector for manipulation by the whites people who run the admissions system. It’s remarkable to me to read anecdotes from admissions counselors, who say stuff like that they are looking for weird and non-conforming kids: https://www.cnbc.com/2021/10/22/former-stanford-admissions-o.... Talk about cultural loading!

Even if you remove personal merit from the idea of college admissions, which most people do not, then you've still got a system that's selecting for the cultural knowledge and position of parents instead of the benefit received from educating a particular person. This seems to be working well enough so far, but it isn't optimal.

I never said that someone's income quantile was dispositive. If my parents never took the SAT and have no idea what it is, I'm at a huge disadvantage to do well on the test by any measure. Regardless of their income. Again, the SAT isn't testing for academic aptitude but for the cultural knowledge of parents and peers.

As a wealthy parent I know exactly how to throw money at prepping for one test: hire expensive tutors and purchase practice tests. Gaming years of GPA or coming up with a compelling story is more complicated and difficult to solve with the application of financial resources.

A system with vague and subjective criteria is subject to bias, for sure. They are also more difficult to manipulate directly with wealth (other than bribes). We have to decide what our priorities are there as a society.

With all due respect, the vast majority of us here took these tests. Do any of us remember them containing quizzes on obscure aspects of water polo and equestrianism and sailing jargon, or other things that only rich kids would know about? I remember them being fairly self-contained.

And that's just the reading comprehension. The idea that the math word problems involve cultural bias that would affect performance is utterly ridiculous. "Most studies"... there is no chance that any quality research has been done to support something so laughable.

I will look at any citations you may have, but I am 100% confident that you have been duped.

To be completely fair, I think at least some portions of the SAT are distinctly more difficult without a strong reading background.

You can definitely cram for the grammar / math portions, but the reading comprehension sort of just requires a lot of reading experience. There isn't a rote set of things you can memorize to get a high score, which could arguably be seen as good or bad.

Maybe more practically, we should consider: Do lower income students read less on average? And if so, is the difference greater than their ability to gain access to tutoring services / prep books which can be used for the crammable portions of the exam?

Succeeding in college also requires a strong reading background. The point of the test is to predict college success, not "be fair" (an inherently subjective statement)

This theory would make sense if the hardest questions showed the most racial disparity, but it's actually the easiest questions that do, according to the papers cited by jedberg.

Here is the main study from 2003: https://meridian.allenpress.com/her/article-abstract/73/1/1/...

That 2003 study was done at Harvard.

Here is a writeup of someone who replicated it with more recent test scores: https://www.insidehighered.com/news/2010/06/21/new-evidence-...

"The SAT has been shown to be both culturally and statistically biased against . . . Asian Americans."

Well, I'll be damned. Huh.

From the second article:

> The 2003 study and this year's found no DIF issues in the mathematics section.

The earlier claim about math word problems being culturally biased seems unsupported by the citations.

But will the claim be withdrawn? Hmm, let's find out...

Here is a simple example from an old SAT (1970s):

Runner is to Marathon as XXXXXXXX is to Regatta

If you grew up on the coast in a wealthy family, you have a much higher chance of knowing what a Regatta is and that it involves boat races.

Come on, man. Your opinions about the SAT are based on a single infamous SAT question (of unknown provenance) from many decades ago?

There are no questions like that any more. You really think the people designing the highest-profile standardized test in the country are oblivious to this kind of problem these days? There's a ton of cultural and racial bias screening and filtering that goes into producing these questions now, both manual and statistical. You can look at some sample questions yourself.

Is there a more up-to-date example? The SAT hasn't even had an analogy section at all since 2005.

Being from Cincinnati, I had to laugh at that particular example a little bit, because there's an annual charity "Rubber Duck Regatta" where you pay to sponsor a rubber duck and they dump them all in the river and give a prize to whoever has theirs finish first. When I was a kid, they ran TV ads for it blasting the name out loud about every six seconds, so most students who didn't know the word otherwise would have figured it out. I'm sure there would be statistical analysts looking at the scores for that question by region and scratching their heads.

Yes, only wealthy people know what a sailboat is and have a large vocabulary.

It doesn't say only wealthy people will know what it is, it says they are much more likely to know what it is because they are much more likely to have experienced it first hand.

It would be hard to deny that generally sailboat racing is done by wealthy people.

That doesn’t follow. Poor people are aware of many things they don’t do. You’re wanting to lower standards to help people who don’t need that kind of help to expiate you own status guilt. Poor people may also not have cars. Or have ever flown on a plane or owned an iPhone. But they aren’t necessarily stupid and will probably know what these things are and if they happen to be then they should go to elite colleges.

That is dependent on where you're from, you could be from a poor family and know exactly what a regatta is.

The yearly regatta has always been like the Olympics of Newfoundland

OTOH, You could be wealthy but from Montana. You probably have never witnessed a regatta in your life. You and your wealthy friends will probably be able to answer every question related to a ski holiday however.

Biased against cultures that don't value competence / excellence in mathematics.

> it is really a test of how wealthy you were growing up

Incredible educational resources are available online for free and just a click away on that supercomputer in everyone's pocket.

There's no reason to be restricted to your parents' vocabulary.

That appears to be a purely statistical argument - some questions appear to be disproportionately hard/easy for certain racial groups. That has some value, but it would be nice to see some concrete example(s) of how this bias is expressed.

I posted an example in a sibling comment:

Runner is to Marathon as XXXXXXXX is to Regatta

If you grew up on the coast in a wealthy family, you have a much higher chance of knowing what a Regatta is and that it involves boat races.

As I replied there, this example is very old. There hasn't even been an analogy section since 2005.

Definitely my most hated section. Full of rare, obscure, you-know-it-or-don't words for massive bookworms.

But if you read the papers cited in the GGGP comment, those hard problems aren't the problems that show the racial gap. The hardest problems show the least gap. Which doesn't fit the pat little sailboat jargon theory at all. However it does fit a common pattern, also seen in longevity statistics, where the most exceptional are more similar to one another than the average are.

> the most exceptional are more similar to one another than the average are

Wait, can you elaborate?

Sure! This idea is interesting and may well be original:

As people get older, the mortality rate gaps between different groups get smaller on a relative basis. Like, the ratio between rich and poor 50 year olds' risk of death over the next year might be 3x or more, but as they age, they become more similar, and by 90 the ratio may be more like 1.3x, and after 100 it becomes indistinguishable from 1 (in part because of thin data).

The theory is that the genetics and environmental conditions needed to survive to an old age are fairly specific, so once you're down to those people, they have less variation from one another. The health risk factors that distinguish income and racial groups may mostly impact younger age mortality, and the people that get past those younger ages may have avoided or resisted those risk factors, regardless of race or income.

Similarly here, students who "survive" to do well on hard SAT questions may be fundamentally very similar to one another (sharing a high g-factor), and this may greatly reduce socioeconomic and racial gaps that are seen on the easier questions.

Does this mean the easier questions are biased? Not necessarily, I would argue. The concepts tested in the easy-medium questions are most likely to be intellectually foundational -- you are far, FAR more likely to get obscure rich people jargon in the hard questions than the easy-medium ones. We want people to have the basic intellectual exposure that enables them to do well on those easier questions, and we should reward it.

The problem is not the test, the problem is racially biased failure to educate in the lead-up to the test. This explains the pattern perfectly: the strongest students power through regardless of background, while the weaker are more sensitive to resources they were given. So you see the biggest gaps on easy-medium questions where basic educational background matters most to getting them right, rather than raw talent.

I see. Very curious. Shall we call it “adverse convergence” hypothesis?

Have you explored broader implications of your idea? For example people who survived tribulations of a political career may end up close to each other despite their initial difference in political opinions, and maybe even across different countries. This doesn’t sound quite as convincing, but I’m kind of grasping in the dark trying to find some other examples.

There's probably a name for it somewhere! I want to call it something like... convergence of survivors, broadly construed. (Achievers? Overcomers?)

I don't know of any examples outside of health but health is a big enough topic and closely related enough to education that I think it's likely to have some application.

Certain psychometric studies also support higher sensitivity to environment in the lower economic strata. Not exactly the same but you see the connection.

The studies cited by jedberg are very interesting, I just don't think they show what he thinks they do.

A better alternative to trashing the SAT is to provide an estimated adversity score, as ETS recently began to do, so schools have a sense of what a student may have had to overcome to reach their level of achievement.

Now that I think of it convergence on a trait crucial to survival is a trivial result, though it’s often non-trivial to spot.

Still there is something in the longevity example that’s evading the trivial explanation.

Yes, it's not trivial because evolution stops caring about you after you reproduce.

It's also a little unclear whether we should consider the % difference convergence remarkable, given that mortality at older ages is so much higher and the absolute difference may not show much convergence.

Aside from the fact, repeatedly pointed out, that the analogy section has been kaput since 2005... this would be a hard question, the sort that shows the least racial and income gaps.

No, it's a measure of how well you perform on the subjects tested by the SAT. There are many things that correlate with high SAT scores (e.g., grades, being Asian?), but it is definitely true that you can prep for the SAT without any meaningful amount of money. I don't believe that anybody has ever alleged that SAT scores are a proxy for intelligence, though I suspect there is a decent correlation. In my mind it is a useful, objective, universal measure of how an individual student is able to prepare for and execute on a difficult task.

SAT's are closely correlated with IQ tests: https://slate.com/technology/2014/04/what-do-sat-and-iq-test...

> I don't believe that anybody has ever alleged that SAT scores are a proxy for intelligence

MENSA accepts it for admission to their high IQ club.

> meaningful amount of money

Depends on how much money is meaningful to you. Time has its own costs, especially since an older teenager can work as an alternative.