News for hackers. Stuff that matters.
Re:IQ Isn't Set In Stone, Suggests Study That Finds Big Jump... by Homunculiheaded on Wednesday October 19, 2011 @05:05PM (#3132081)
I love when people ask "what does your IQ score measure". The only thing it can possibly measure is your ability to perform an IQ test. Now you can ask the question "What does your IQ score strongly correlate with", but I'm pretty sure as it stands 'intelligence' is not well defined enough assign it a quantitative value (or even know if one can be defined).

I used to hate standardized tests (and still somewhat do in principle) but when you can show that a test score correlates to performance it does make sense to assist in making decision when there are very large sets of data to process (like college applications). It turns out that SATs do correlate somewhat to freshman year performance. It doesn't really matter if it's for the right reason, if you have thousands of similar applicants it makes sense to use that data point if there are no other options. This is the logic of machine learning too: I don't really care why 2 variables correlate, if they do it can help me make decisions.

The issue for me is that I have yet to see anything interesting about what IQ really correlates to. Pretending it is a test that measures intelligence is completely asinine if you take 2 minutes to think about what that claim means.

Re:IQ Isn't Set In Stone, Suggests Study That Finds Big Jump... by bad_user on Wednesday October 19, 2011 @05:34PM (#3132210)
I think it is generally understood by psychologists, sociologists and statisticians that intelligence is not something that you can "measure". Nobody in the know even pretends that.

What happens is that people would rather put faith in the correlation between IQ scores and intelligence, rather than nothing at all. It's annoying as hell as IMHO a +/- 15 points doesn't mean anything, but that doesn't stop people making decisions based on that. On the other hand you can clearly say that somebody with an IQ of 240 is gifted or that somebody with an IQ of 60 is challenged.

The problem with SAT scores is that people doing well on it are the people fresh out of high-school. You cannot give a SAT test to somebody that's been 10 years out of high-school or to a 10 year old.

Re:IQ Isn't Set In Stone, Suggests Study That Finds Big Jump... by byrneseyeview on Wednesday October 19, 2011 @06:17PM (#3132368)
I'll take the other side of that bet!

What kind of odds would you give me that a group of people in the 10th percentile will have a higher average income than a group of people from the 90th percentile? If all IQ measures is your ability to take an IQ test, I think you'd be pretty confident that the bottom 10% will perform about as well as the top 10%. So, would you up up for 1:1 odds on that?

If you're willing to bet consistently with your beliefs, please get in touch; my email address is in my profile.

Re:IQ Isn't Set In Stone, Suggests Study That Finds Big Jump... by Homunculiheaded on Wednesday October 19, 2011 @06:49PM (#3132471)
I think you're confusing measures X and correlates with X. Snowfall probably correlates very strongly with the number or people skiing on a weekend, but it does not measure the number of people skiing on a weekend.

Let's say you win this bet (which I wouldn't take since I never argued that was the case in the first place), all you have shown is that the ability to answer questions on an IQ test has a strong correlation with income. How does this have anything to do with intelligence, which the test claims to measure? If you where to say that "well this means that intelligence is strongly correlated with income" this is tautological since the IQ test is your only way to measure intelligence in the first place.

My assumption that IQ scores probably don't correlate that strongly to many things is based on the observation that we don't use it that much. If you look more thoroughly there are, as expected, obviously correlations between IQ and many things, but how strong those correlations are and the variance make many of them interesting but not fantastically useful.

Correlating with the extremes is not that interesting, now if you where to show that the is a strong correlation between IQ test percentile and income percentile across the board that would be interesting. If you're really interested there has been research into the correlation between IQ and Income and it's about 0.4-0.5 [1]

1. http://en.wikipedia.org/wiki/Intelligence_quotient#Income

Re:IQ Isn't Set In Stone, Suggests Study That Finds Big Jump... by Steko on Wednesday October 19, 2011 @06:31PM (#3132426)
"If all IQ measures is your ability to take an IQ test, I think you'd be pretty confident that the bottom 10% will perform about as well as the top 10%."

Unless the top 10% has some systemic advantage at taking IQ tests. Type words into a search engine for further inquiries in this area.

Re:IQ Isn't Set In Stone, Suggests Study That Finds Big Jump... by byrneseyeview on Wednesday October 19, 2011 @06:35PM (#3132442)
Doesn't that seem kind of redundant? "People who do well on IQ tests have systematic advantages which allow them to do well on IQ tests, but IQ tests are meaningless."

Obviously, it's possible to imagine a world in which IQ is not largely heritable, and in which rich people pay lots of money to game their kids' IQ numbers, and then the kids somehow use these IQ test numbers to get better jobs (even though IQ testing from employers is basically illegal).

But we can adjust for that. I'd be happy to do the same bet controlling for parental socio-economic status.

Re:IQ Isn't Set In Stone, Suggests Study That Finds Big Jump... by Steko on Wednesday October 19, 2011 @06:55PM (#3132500)
"it's possible to imagine a world in which IQ is not largely heritable, and in which rich people pay lots of money to game their kids' IQ numbers, and then the kids somehow use these IQ test numbers to get better jobs"

You don't need to imagine that world, we live in it. Kids who go to private schools with SAT prep classes and practice the test dozens of times are at a several hundred point advantage to kids who don't. This gets them into better colleges which gets them better jobs and gets them better income.

Re:IQ Isn't Set In Stone, Suggests Study That Finds Big Jump... by throwawayday on Thursday October 20, 2011 @10:05AM (#3134857)
I beg to differ: I grew up in a trailer park, with an alcoholic carpenter for a dad. We certainly didn't have the money to send me to prep classes or private school. I did have one thing going for me: a love of books. I read more books than an whole classroom of kids at my school. As a result, my SAT scores were in the top 1%. I make well above the average now, and my family enjoys the type of lifestyle that I could only dream about as a child.
Re:IQ Isn't Set In Stone, Suggests Study That Finds Big Jump... by Steko on Thursday October 20, 2011 @03:56PM (#3136614)
Observing that preparation generally improves scores =/= you can't score well without preparation. Moving the bell curve a little to the left or right doesn't cut off the tail on the other side.
Re:IQ Isn't Set In Stone, Suggests Study That Finds Big Jump... by Retric on Wednesday October 19, 2011 @07:32PM (#3132642)
The SAT is not an IQ test, it goal is to measure the probability of someone finishing their freshman year of collage and when adjusted for the collage it's better at that than HS grades.
Re:IQ Isn't Set In Stone, Suggests Study That Finds Big Jump... by Steko on Wednesday October 19, 2011 @07:47PM (#3132687)
No it's the worst predictor of finishing freshman year. The claim is that SAT + high school GPA is stronger then GPA alone which is a pretty weak claim.

As for what it actually measures (as opposed to what it trys to measure), opinions differ...

http://en.wikipedia.org/wiki/SAT#Correlations_with_IQ

"Frey and Detterman (2003) analyzed the correlation of SAT scores with intelligence test scores.[20] They found SAT scores to be highly correlated with general mental ability, or g (r=.82 in their sample). The correlation between SAT scores and scores on the Raven's Advanced Progressive Matrices was .483 (.72 corrected for restricted range). They concluded that the SAT is primarily a test of g. Beaujean and colleagues (2006) have reached similar conclusions.[21]"

"Certain high IQ societies, like Mensa, the Prometheus Society and the Triple Nine Society, use scores from certain years as one of their admission tests. For instance, the Triple Nine Society accepts scores of 1450 on tests taken before April 1995, and scores of at least 1520 on tests taken between April 1995 and February 2005."

Re:IQ Isn't Set In Stone, Suggests Study That Finds Big Jump... by scott_s on Thursday October 20, 2011 @09:36AM (#3134726)
I took a course with Clark Glymour when he was visiting my school. His speciality is trying to determine if we can infer causation. (See: http://www.hss.cmu.edu/philosophy/faculty-glymour.php)

At one point, the administration at CMU came to him and asked if he could figure out what was the best predictor of students finishing college. (Or it might have been just freshmen year. It's been a few years since that course.) He had access to all of CMU's data on students. The result: SAT scores.

Whether or not the SAT itself is a predictor of actual intelligence is besides the point.

Re:IQ Isn't Set In Stone, Suggests Study That Finds Big Jump... by Steko on Thursday October 20, 2011 @04:11PM (#3136700)
I haven't seen anyone claim it's the best predictor of college performance. Some say it's an ok predictor, some say it's near useless, a good summary of different sides of this can be found here:

http://answers.google.com/answers/threadview/id/592879.html

Re:IQ Isn't Set In Stone, Suggests Study That Finds Big Jump... by Retric on Thursday October 20, 2011 @09:03PM (#3137842)
There is a huge gap between saying SAT's relate to finishing your freshman year and your GPA. Granted finishing your freshman year depends on GPA, but it's more a question of doing the basics. AKA doing your homework without mommy and daddy there, handing things in on time, not cheating, going to class and those activity's are far more independent of HS GPA than parents want to admit.

On the other hand GPA's over 3 depend a lot on how hard your classes are, how heavy the work load, if you have a full/part time job which are less dependent on the student's innate qualities.

Re:IQ Isn't Set In Stone, Suggests Study That Finds Big Jump... by byrneseyeview on Friday October 21, 2011 @11:03AM (#3140033)
Right, but there's a glaring statistical error. Schools are very stratified by skill level. So you may have someone with a 1550 and a 2.5 HS GPA, sitting next to someone with a 1300 and a 3.8 HS GPA. One smart slacker, one hard worker who isn't quite so facile.

Of course SAT scores won't be predictive in that case. You'd want to look at SAT scores controlling for other admissions factors, which is basically SAT scores controlling for every variable admissions cares about, which of course means that no variable in particular has predictive power.

But if you want to see how strongly SAT scores correlate with academic performance, the way to do it would be to take some students who belong at, e.g., ASU, and somehow sneak them into Harvard.

Re:IQ Isn't Set In Stone, Suggests Study That Finds Big Jump... by Retric on Wednesday October 19, 2011 @08:13PM (#3132757)
It's seen many changes over time, but gave up the idea of being an intelligence test over 20 years ago.

The name originally stood for "Scholastic Aptitude Test".[35] But in 1990, because of uncertainty about the SAT's ability to function as an intelligence test, the name was changed to Scholastic Assessment Test. http://en.wikipedia.org/wiki/SAT#History

Re:IQ Isn't Set In Stone, Suggests Study That Finds Big Jump... by Steko on Wednesday October 19, 2011 @09:28PM (#3132973)
Spoilers: I don't think SAT measures "intelligence" any more then IQ tests do. That's what this sub-thread is about - IQ tests don't measure what they purport to either.
Re:IQ Isn't Set In Stone, Suggests Study That Finds Big Jump... by vacri on Wednesday October 19, 2011 @05:36PM (#3132225)
A real IQ test is a decent measure of general intelligence. It does, however, take a trained professional several hours to administer - precisely because this is necessary to counter most of the criticisms levelled at "IQ tests are crap". Components of the IQ test can also be used to track individual aptitudes. But because it's so resource intensive to conduct, pretty much any time you've heard someone say "My IQ is -foo-", you can be fairly sure it's not from the formal test.

Everything else that calls itself an IQ test is really a form of pop science quiz and of course has flaws. Any test that you're supposed to administer yourself is not the IQ test - that's the easiest first step.

Re:IQ Isn't Set In Stone, Suggests Study That Finds Big Jump... by Homunculiheaded on Wednesday October 19, 2011 @06:14PM (#3132358)
but how do you quantify intelligence in order to determine that there is a relationship between it and an IQ test score? If you can say that a real IQ test is a decent measure of intelligence, how to do you know this?

Let's say I have a Teakettle test which assigns a score to the temperature of the water in a teakettle based on only qualitative properties of a teakettle (is it whistling, is it rumbling, does touching it cause pain?). This test is probably good at assessing the extreme cases (water is 212 F, water is around 70F) but I'd guess it's a poor predictor of temperature. Now if you gave me a thermometer I could see how correct my test was.

The problem is we don't have a thermometer for intelligence (that I know of) and more important I don't know if there is any consensus that 'intelligence' can even be assigned a quantitative value the same way the vibration of molecules can.

Now maybe the IQ test, like the Teakettle test, is a good summation of qualitative properties. Now you can compare this quantitative value to others to show a useful correlation. The Teakettle test may be a really great way to predict time to tea-time. An IQ score can only be useful if it shows a strong correlation to other, quantitative things (SAT score, salary, life expectancy, number of hairs in eyebrows)

To say that a real IQ test is a decent measure of intelligence seems to be at best a tautology.

Re:IQ Isn't Set In Stone, Suggests Study That Finds Big Jump... by vacri on Wednesday October 19, 2011 @08:17PM (#3132771)
The IQ test is multivariate and has a number of axes which are not boolean like your teakettle test is.

Intelligence is not a property like physical attributes. It is fuzzy and has a lot of aspects. Some forms of intelligence aren't tested with the IQ test (unless somehow done recently), the primary one that comes to mind is social nous.

The psychology of intelligence is a complex, varied, and interesting field, and those who naysay "you can't study / define / measure intelligence!" are submitting to pop science just like "take this pop IQ quiz!". I've lost track of it over the years, but there are reams of research into intelligence, what it is, how it's defined, how tests do or don't reflect intelligence.

The IQ test covers a lot of different aspects of intelligence - it gives a fairly comprehensive representation of many aspects, and hence why it's a decent measure of intelligence. It's not a cut-and-dried thermometer measure, yes - but this is because psychology never works that way, something that engineers have trouble getting their heads around. Nothing in psychology is concrete, and if you're going to dismiss based on not being 100% precise, well, then psychology can't exist, nor can any of the fields based from it (like sociology, criminology, so on and so forth)

Re:IQ Isn't Set In Stone, Suggests Study That Finds Big Jump... by gwern on Wednesday October 19, 2011 @07:30PM (#3132632)
> It does, however, take a trained professional several hours to administer - precisely because this is necessary to counter most of the criticisms levelled at "IQ tests are crap".

Don't the shortened IQ tests correlate pretty strongly with the longer ones? When I read psychology papers, even ones dealing with children, they pretty frequently use speeded or abbreviated ones without apparent qualms.

Re:IQ Isn't Set In Stone, Suggests Study That Finds Big Jump... by vacri on Wednesday October 19, 2011 @08:21PM (#3132784)
Correlate well, yes, but a full test will show a better spread of aptitudes.

I did a partial IQ test when I was studying intelligence. I was on track for getting around 140 according to the conductor of the test. But I can't say my IQ is 140 because I may have fared worse (or better!) on other parts of the test (for example, we didn't even touch on the verbal tests).

Re:IQ Isn't Set In Stone, Suggests Study That Finds Big Jump... by esrauch on Wednesday October 19, 2011 @07:08PM (#3132542)
> I love when people ask "what does your IQ score measure". The only thing it can possibly measure is your ability to perform an IQ test.

This is a completely random statement. You could just as easily say that an analog scale doesn't measure weight, it measures how much a spring compresses when an object is placed on top of it. Since the spring is different every time, even the same scale is not going to be the same every day.

Measuring something and what it correlates with is exactly the same thing, and the distinction you are making is absurd pedantry.

I don't really have any strong opinion about your actual stances.

Re:IQ Isn't Set In Stone, Suggests Study That Finds Big Jump... by Homunculiheaded on Wednesday October 19, 2011 @08:00PM (#3132723)
Weight isn't some property we just defined based on the compression of a spring, and we aren't measuring just a correlation. When we measure weight on an analog scale we already know several, quantitative physical properties of the spring, we also have a mathematical model of how exactly the compression of a spring is directly related (not correlated) to the force of weight. Given that we know all of the additional variables of a spring we take it's compression and determine the weight. Inaccuracies in analog scales are not due to some stochastic process involved with the correlation between weight and compression, but inaccuracies measuring the quantitative properties of the spring.

In an IQ test we are merely counting a set of qualitative properties (answered question n correctly). There is no underlying model that we are using to determine "intelligence", there is nothing quantitative about the IQ test other than counting the correct answers. We don't even have a quantitative model of intelligence at all, so there is simply no way to measure it. It is not pedantry to say that what an IQ test does is merely measure answered questions and not intelligence. If there where a.) a quantitative model of intelligence, and b.) a formula mapping this to IQ tests then, yes we could say we were measuring intelligence in the same way an analog scale measures weight.

Re:IQ Isn't Set In Stone, Suggests Study That Finds Big Jump... by esrauch on Wednesday October 19, 2011 @11:08PM (#3133252)
> what an IQ test does is merely measure answered questions and not intelligence.

I get your point, but I think it is still not quite a cut and dry as you are making it. The issue is not that intelligence is something quantifiable for which they haven't been able to map IQ on to; intelligence is just an ill-defined concept. Absolutely nothing can possibly measure or map onto intelligence because there just is no real definition of the word.

Saying that IQ measures intelligence is the same as saying that answering questions on a standardized test requires intelligence, which seems like a perfectly reasonable and valid definition to me.

Would you bring up this argument someone said "He solved that math problem that has been open for hundreds of years, he is intelligent". Would you jump in and say "actually, solving math problems isn't necessarily a measure of intelligence, which is what you are implying".

All that solving famous unsolved math proofs shows is that you are better at solving math proofs that a bunch of other people who try to solve math proofs, what does that have to do with intelligence!

Re:IQ Isn't Set In Stone, Suggests Study That Finds Big Jump... by Homunculiheaded on Thursday October 20, 2011 @12:04AM (#3133400)
And I think I'm starting to get your point as well ;) Perhaps I would have been more explicit. The key point here is the difference between qualitative and quantitative data. "intelligence" as we know it is a qualitative property. So it is perfectly okay to say "He solved that math problem... he is smart", I'm honestly not being a pedant here. There is a language and method for discussing qualitative properties of things: most of philosophy deals with these issues, the arts deals with these, and the majority of everyday language also deals with the qualitative. So depending on the degree of formalism, expressiveness and seriousness you want to discuss something qualitative we have a framework for that.

What you cannot do, despite any amount of manipulation and/or trickery, is talk about qualitative data as though it were quantitative. One very important aspect of quantitative data is that there is an ordered relation on a given set of quantitative data, so you can make absolute and accurate claims about how that data is ordered. One person's height can be quantitatively shown to be higher than anothers, their weight larger, there top speed faster. And all of these properties can be shown to be undisputedly so, that's the nature of quantitative data.

And here's they key point: you can absolutely show that someone's IQ score is higher than anothers, but you cannot absolutely show that someone is more intelligent than another. There is not an ordered relation on the set of human intelligence.

Now I know what you might say "But surely someone with an IQ of 70 and and IQ of 170 can be shown to be different levels of intelligence". But these are extremes and I would argue you're still ultimately making an assessment based on obvious, qualitative properties of intelligence.

Take for example the property of "friendliness", which is qualitative. Now you and I can could probably come up with some ordering among people in a room as to who was the least and most friendly. But if we moved to different cities or countries that ordering might change. I would bet our extreme case would still hold, that is the person we decided was the least friendly, would probably (but not absolutely) be seen as unfriendly in many environments. We can compare to a degree qualitative data, but it is always fuzzy, and never absolute.

This is so important, and more than just pedantry, because it is a logical error to try to compare qualitative things with quantitative means. It leads to a variety of poor/wrong conclusions about data and situations. This is why it is essential to realize that the number of correct answers on an IQ test can be correlated to a variety of other quantitative data. We may associate much of this quantitative data with qualities of intelligence, but this is by no means the same as making the claim that we are measuring something which is, by our current definition, immeasurable.

Re:IQ Isn't Set In Stone, Suggests Study That Finds Big Jump... by jpulgarin on Wednesday October 19, 2011 @05:19PM (#3132149)
Correlates somewhat to academic performance, lifetime earning, SAT scores etc...