Re:The Oxford maths interview, shaking up admissions
by Houshalter on Thursday March 03, 2016 @04:25PM
(#11220166)
This is just speculation but I have some ideas of what effects might be in play here.
First the exams test knowledge, not problem solving ability, which is what he wants. It may be possible to design a test for problem solving ability. That's basically what IQ tests are designed to do. As I understand it, the SAT test used to be like an IQ test and highly correlated with IQ.
Second exams probably do correlate with IQ (it correlates with literally everything), but correlation isn't enough. You can't just take the top n results from a test. You can use a test as a filter, e.g. sampling the top 5% of results or whatever. But when you set the filter too high, you just get outliers.
E.g. the top 100 people who are freakishly good at taking tests, and nothing else. Maybe they have abnormally good memories and just remember everything they read in the textbook. Or maybe they have parents that pushed them to study excessively, or just did so on their own.
You only should use tests as a filter, a minimum standard. Not optimize for it directly.
First the exams test knowledge, not problem solving ability, which is what he wants. It may be possible to design a test for problem solving ability. That's basically what IQ tests are designed to do. As I understand it, the SAT test used to be like an IQ test and highly correlated with IQ.
Second exams probably do correlate with IQ (it correlates with literally everything), but correlation isn't enough. You can't just take the top n results from a test. You can use a test as a filter, e.g. sampling the top 5% of results or whatever. But when you set the filter too high, you just get outliers.
E.g. the top 100 people who are freakishly good at taking tests, and nothing else. Maybe they have abnormally good memories and just remember everything they read in the textbook. Or maybe they have parents that pushed them to study excessively, or just did so on their own.
You only should use tests as a filter, a minimum standard. Not optimize for it directly.