News for hackers. Stuff that matters.
Re:MIT to no longer consider SAT subject tests in admissions... by austincheney on Friday March 20, 2020 @03:06PM (#22640540)
The problem with tests like the SAT is that on one hand they produce quantifiable results and the comparison of one student's score to another student's score is indeed objective, but that does not mean the test itself is an objective measure of performance.

The primary fault is that the SAT is essentially a form of convergent IQ test. A convergent test is a test of questions with the accepted answered defined before the test attempt and comparing the test takers answers to the defined answers. This is convergent, or coming together, in that performance is measured against subjective, subject based (inferred acceptance), criteria. I understand the SAT is intended to be used as a measure of general scholarly assessment, but its primarily used as a discriminatory filter to limit access to a preferential segment. As such it is essentially an arbitrary IQ test in practice regardless of its intentions.

In contrast IQ tests more generally preferred by the psychiatric community tend, almost exclusively, are divergent tests. A divergent test has questions without any prior identified answer and so there is not a simple right/wrong conclusion. Instead the tests generally seek out things like abstract reasoning, creativity, reasoning, answer diversity, and other aptitude performance criteria.

In the few psychiatric administered aptitude tests I have taken I generally test a bit below the genius level. If on a numeric IQ scale genius is 150 or 155 I would be below that at 140 to 150 depending on the test and test criteria. The SAT, on the other hand, rated me as outstanding a math and reasoning but otherwise barely literate. Those SAT scores are clearly at odds with my real world performance and other test results.

Confusing the potential for objectivity, such as comparing score results, to actual objectivity, such as whether the scores are a valid measure in the first place is a common error. The most common IQ test is the Stanford-Binet Test formed by Dr. Lewis Terman who made the same error:

* https://www.psychologytoday.com/us/blog/beautiful-minds/2009...

* https://www.latimes.com/archives/la-xpm-1992-05-31-bk-1247-s...

Re:MIT to no longer consider SAT subject tests in admissions... by knzhou on Friday March 20, 2020 @03:13PM (#22640618)
I'm sure the writers of the SAT are well aware of this, but full IQ tests just don't scale. How are you going to provide the same test ten million times a year, when the test has to be administered one-on-one by a trained professional over the course of hours? You're going to have to hire tens of thousands of administrators, so how are you going to keep bias from creeping in? How are you going to prevent the questions from leaking, or keep people from rehearsing scripts on the subjective parts? IQ tests have never dealt with these problems only because they haven't had to scale, not because they're immune to them.
Re:MIT to no longer consider SAT subject tests in admissions... by austincheney on Friday March 20, 2020 @03:29PM (#22640751)
> I'm sure the writers of the SAT are well aware of this, but full IQ tests just don't scale.

First of all its generally faulty to presume what others are thinking.

Secondly, standardized admissions testing in its current form should be eliminated and there is a fair amount of data to support this[1][2]. If schools really want to discriminate on performance they should test for originality, composition, and decision capacity all of which can be done objectively with automated administration.

[1] https://www.forbes.com/sites/kimelsesser/2019/12/11/lawsuit-...

[2] http://www.nea.org/home/73288.htm

Re:MIT to no longer consider SAT subject tests in admissions... by knzhou on Friday March 20, 2020 @03:43PM (#22640903)
Sorry, but I write tests and I have absolutely no idea how you propose to measure these things objectively and automatically, without producing exactly the kind of "convergent" test you said you didn't like. For example, take "decision capacity". I don't even know what you mean by that, and I don't know how this will be assessed automatically without fixing a pre-designed correct answer.

Incidentally, the GRE has a system for grading essay composition automatically. It also absolutely sucks. It has no idea what your arguments are actually saying, so points are allocated based on irrelevant features like average sentence length or the total number of paragraphs. Researchers have succeeded in getting perfect scores by just copy-pasting the same sentence 25 times. In my book, this is worse than multiple choice.

Re:MIT to no longer consider SAT subject tests in admissions... by Animats on Friday March 20, 2020 @04:50PM (#22641520)
For example, take "decision capacity".

For that, there's an "inbox test", a set of incoming messages to be dealt with in order. Such tests are widely used to screen candidates for manager jobs.

Re:MIT to no longer consider SAT subject tests in admissions... by knzhou on Friday March 20, 2020 @05:28PM (#22641831)
Wow, I just learned something new. These look legit and well-matched to the actual work. It does seem hard to make a "standard" decision test though, it seems each would have to be tuned to the job it was made for.
Re:MIT to no longer consider SAT subject tests in admissions... by WalterBright on Friday March 20, 2020 @03:42PM (#22640897)
SAT scores are used because they turn out to be good predictors of success at college. They aren't the only measure used, probably about a third.

For example, my SAT scores were average for an MIT admission, but I was rejected.

Re:MIT to no longer consider SAT subject tests in admissions... by austincheney on Friday March 20, 2020 @04:12PM (#22641170)
There is a lot of evidence that this is absolutely not the case. On of my links in a prior comment indicated there are now many universities that are standard test optional and keep stats comparing the performance between standard test students versus other students to find no difference in performance.

Consider your example. You are a famous developer well known for a performance oriented programming language. Of the many computer science students that graduated from MIT I cannot name any who are as well known for such. It is impossible to say that if your SAT scores were closer to perfect you might have gained admission. Regardless you have clearly excelled where others have not regardless of institution or institutional entry. At the least this suggests the incorrect combination of parameters were assessed given real world performance.

Re:MIT to no longer consider SAT subject tests in admissions... by WalterBright on Saturday March 21, 2020 @03:28AM (#22645165)
My degree is in Aerospace Engineering, not CS :-)

Though I shamefully admit my lack of academic CS knowledge shows its face now and then. I sometimes find myself reinventing the wheel.

Re:MIT to no longer consider SAT subject tests in admissions... by leetcrew on Friday March 20, 2020 @05:16PM (#22641735)
I don't entirely understand your objection. the "meat-and-potatoes" skills tested by the SAT seem to be english vocab and algebra. there's certainly some gray area in the meanings of english words, but the examples on the SAT tend to be pretty cut and dry. as for the basic algebra questions, I don't really see how it could make sense for the answer not to be determined beforehand. 5 will always be the x value that make 5x = 25 true.

seems like as good a way as any to predict whether you'll be able to handle the rigor of 100-level courses in your freshman year. basically answers the question: "can you read and understand what we plan to assign?"

edit: out of curiosity I perused some of your other posts. it seems like you have a pretty good grasp of english vocab and sentence construction. I'm not really sure how you could have bombed the critical reading and writing sections.