Are you suggesting that having people capable of scoring well on exams of common necessities for college education, like mathematics and language skills, is not desirable? The SAT is not a trivia test: a student who can pass a test on math up through precalculus is going to get more value out of college than one who does not understand algebra and has to spend time in remediation. I would much prefer to have a university exclusively populated by people good at solving math problems and comprehending readings than by people selected for their "good vibes" and to fit a diversity quota.

You didn’t address the comment you’re responding to. You won’t have students who are good at math and language skills. You’ll have students who are good at taking math and language skills standardized tests. Being good at math and being good at taking a math standardized test is two different things. A student can excel at math but be terrible at taking tests for many reasons. You’ll have bright students falling through the cracks if this was the only consideration for admission.

How do you know that this hypothetical student “excels” at math if their performance can’t be measured? Students can and are already given reasonable accommodations. Sure tests may be imperfect but I am highly skeptical of any claim that math test performance and academic math performance are not highly correlated.

At the levels we’re talking about, the difference between “perfect score” and “still excellent but imperfect score” can be down to one or two errors on the test, basically noise. Given the number of applicants and the willingness to drill testing skills, going by scores would eliminate anyone who makes a minor error. The ability to perform perfectly in high-stress situations is a useful (and learnable) skill - like the ability to win quiz games, or perform classical music at concert level - but it is not the same thing as academic potential.

(As an aside: as an advisor of PhD students in computer science, I sometimes have to train students out of these habits. Research doesn’t happen if you’re used to getting the shallow answer more quickly, or if you’re afraid to be wrong.)

Yes, and that’s one of the reasons why tests are imperfect. That doesn’t mean that introducing any additional variables is warranted, especially given their associated noise. GPA is garbage, for example, which is why I ignore it on resumes. If the test has unacceptably high variance as you mentioned, the solution is to make the test longer not discount it.

I see no evidence that making tests longer will optimize for the variables that matter in academic performance (which are not 1-dimensional anyway) as opposed to simply optimizing for people who have poured resources into test preparation.

You seem to be coming from the perspective that testing must be the best metric, and then trying to develop a methodology based on that assumption. Purely as a thought exercise: it might be helpful to start from the assumption that testing isn’t an option, and think about what you might do instead.

Hard disagree about it being possible to be good at math but having trouble with something like SAT math.

Any student that can do any sort of undergrad level math like AP Calculus is going to find SAT math trivial in comparison. There's no way you can pass exams in the first but do poorly on SAT math, since all the stuff SAT math is testing on are prereqs for something like calculus.

I took the SAT decades ago, so the predictive nature of the SAT may have changed since — but I doubt that it has changed that much.

At that time, the SAT was good at predicting the likely performance of a prospective college student … for one term. That’s it. I got an excellent score on my SAT (not a perfect score, but I thought that it was a bullshit measure — which, like all standardized testing, it is — so I didn’t care). I did well in my first term, and then not so well after that (I ended up doing fine in the end, but it took longer to get through a degree program because I didn’t know what I wanted to be when I grew up; I think that Americans, by and large, put too much focus on going to university too early and we would be better served by an extra year of high school and/or a gap year).

One other thing I remember from that time period is that a better predictor of overall success in school was NOBITH, the Number Of Bathrooms In The House. That is, socioeconomic indicators were a better predictor than anything else, including nonsense standardized tests like the SAT. And given that socioeconomic indicators are a better indicator, that is a strong indication that there’s seriously wasted potential at all levels of eduction because we mistakenly equate family socioeconomics with success in general.