I mean, the SAT has "aptitude test" in the name; the idea that you can practice it and do better each time is a bug and not a feature.

But I think the bigger danger is that after a while, practicing for standardized tests starts monopolizing the curriculum at secondary schools, and teachers start being evaluated by their students' test scores to the exclusion of all else. Even 20 years ago, I remember getting herded onto a bus to some community college to take an extra "practice state proficiency test" whose score didn't count for anything at all.

Even if universities have lousy motivations, de-emphasizing that kind of thing probably makes high-schoolers' lives a little better.

Since 1993 SAT no longer stands for 'scholastic aptitude test'. It had its roots as a sort of IQ proxy but that has fallen out of favor because it doesn't reflect the full purpose of schooling (e.g. advancement from given means, value-add of schooling for a given student) nor range of things that cause students to achieve (e.g. hard work, conscientiousness, etc).

One could make a test called 'big brain test' and sell it to schools to say it measures big brains, and it might be only somewhat correlated but still get widespread adoption because it fulfills a need schools have to have some sort of quantitative sorting of students, but it still wouldn't necessarily be a measure of big brains. This is sort of what the history of the SAT is.

I'm not aware of any significant cultural test bias (which I'll define as differential prediction).

I'm an immigrant and moved here when I was 9. My parents always spoke in a different language growing up. I personally had a hard time with certain vocabulary that I never picked up and that maybe I otherwise would have if I had native English-speaking parents. This definitely slows you down when reading problems as you're only gathering about 3/4 of the problem at times. This is probably an outlier, but just thought I'd share my point of view here.

Statistically significant, such that it created some academic literature and enough cover for Harvard to make this new policy. Practically significant, maybe not.

Here's a comment from a different branch of this thread:

> The findings in general is that much of the disparity between minorities and whites goes away for harder analogies that are more technical or scientific in nature, in fact contrary to what I expected, the recommendation of the study is to simply eliminate the bottom third of verbal questions (ranked by difficulty) involving analogies or sentence completion, as those are the questions that are most likely to involve cultural bias.

https://news.ycombinator.com/item?id=29599455#29600100

First off, the SAT has gone through many revisions in the last 20 years so it's unclear this still applies. The SAT doesn't even have analogies any more, perhaps as a result of this research.

Secondly, that doesn't address differential prediction abilities. It addresses a concerning aspect of the verbal test, but is somewhat irrelevant when your goal is to predict college success. I'm not aware of differences on that metric.

> The SAT doesn't even have analogies any more, perhaps as a result of this research.

That's good to hear. It's hard to know what research is still fresh, unless one is employed to stay current in a field.

What’s wrong with analogies?

Many analogy questions suffered from requiring cultural context, and it's hard for a question-writer to know which ones.

Biased in the sense that cultures that that rewarded diligence and intelligence are able to compete more effectively against cultures that don't and the wages of these accomplishments can be seen in their posterity?