Of course Taleb understand statistics if he wants to, but he is just pushing his rants and loses his bearings in Twitter. Even his old friends are lamenting his online personality and bitterness.
A good portion of studies don't replicate, including fundamental ones (in particular see the failed replication here [1] on one on Rand et al, a study on the effects of priming). Priming is a huge topic in social psychology.
He understands statistics. I'm not sure his opponents properly understand that when you complain about methodology, the implications are not just "add a section in the paper acknowledging the potential for error", it's "your whole study might be fatally flawed in a way that invalidates all your conclusions".
[1]: https://authors.library.caltech.edu/91063/2/41562_2018_399_M...
There is a big body of literature on priming. Each study is generally done to get a p-value < 0.05. In a sense, there are a bunch of replications of the effect itself. That points to priming as an effect large enough to matter.
There is another viewpoint, where priming is not an effect large enough to matter. (This is the viewpoint I hold.) The arguments for this viewpoint are that the original study does not replicate - the 2018 replication attempt I linked used a ~300% larger sample size (1014/343), but achieved a p-value of 0.366 and had an effect size 80% smaller than the original. A second argument is that priming is not used in industry, though the effect would be useful in fields like advertising or military psyops. A third argument is that there is a widespread suspicion in the field that psychology researchers are p-hacking to get spurious results.
A whole subfield exists on an effect that showed an 80% reduction in effect size with a 300% larger sample size and a 4000% increase in p-value on a direct replication. And my focus on this study ignores the fact that the group of replications turned up 9 failures in 21 replications pulled exclusively from studies picked from Nature and Science.
If psychology can botch the literature on priming this badly, what else have they botched?
Is that claim true somehow? It certainly looks wrong, looking at the scatter plot (even setting aside whether a linear regression is appropriate). Are scatter plots 'bad methodology' somehow?
(BTW the comparison of IQ to priming is ridiculous. IQ is the most replicated and reliable measure in all of psychometrics.)
This is not a high barrier to pass. Psychometrics is a field notorious for employing biased and non-existing constructs.
Just because it is well studied, doesn’t make it real. N-rays were at one point one of the most studied rays in physics. It didn’t make measurements using n-rays more reliable. If everyone is repeating the same mistake, it doesn’t erase the mistake.
You know the history of IQ and how countless studies in the past are biased in horrendous ways. It might be well replicated and "reliable", it is still wrong.
If you want to argue that IQ is "wrong" you need to explain https://www.ncbi.nlm.nih.gov/pubmed/8162884 and every other result that has been published relating to it, not just make vague insinuations that IQ is just like social priming (it's not).
I don’t know why you are all of a sudden talking about social priming (I’ve never seen it pop up before in this thread), especially since priming is a concept from physio-psychology and computation psychology (that has apparently been borrowed by social psychologists) not psychometrics. I’m not even sure what you mean by social priming (I did a quick browse on wikipedia without success) so you have to inform me.
If you are saying that my claim—that psychometrics is a field filled with pseudoscience—is unsubstantiated, you are right. I did (implicitly) claim that, and I didn’t provide any substantial evidence for my claim. I probably should have, but that is out of the scope of this thread, so I’ll just leave it unsubstantiated. Call me lazy, and you would be correct.
---
Edit: To clarify. Priming did come up in a grandparent’s comment. Priming (as my layman understanding goes) is believed to be a neurological effect that increases the efficiency of a search response for similar stimuli presented at short intervals. That is finding a particular pattern gets easier with subsequent trials. Priming effects have been demonstrated in numerous studies in the past two decades. However (as is usually the case in many scientific fields) a hype has arisen around the concept and many scientist are claiming that priming can explain several unrelated psychological constructs. Many of these studies have poor methodology and have never been replicated. Perhaps my parent comment was talking about one of these studies when they mentioned “social priming”.
Yes, social priming studies are what cljs-js-eval was referring to originally when they mentioned "priming". Priming itself is generally solid science (eg. the Stroop effect).
Liberals are happy to question IQ and especially race/IQ work while conservatives are happy to question stuff like those gender blinded recruitment studies or those academic trolling studies that try to link conservative opinions with mental illness.
The reality is that all of it is very questionable because the entire field is riddled with shaky methodology and down right bad science. From what I've seen of the replication issues the whole field is worse than nutritional science, and that's bad.
The degree to which a scientific field is politically weaponized is usually inversely proportional to its "hardness." You don't see the same thing in math or physics. Liberals and conservatives oddly never disagree on the value of Pi or the formula for the Carnot efficiency of a heat engine. The closest things to hard science that you find massive political disagreements on are climate change and evolution, and I've noticed that more serious conservative thinkers are coming around on those topics because the evidence is overwhelming.
https://en.wikipedia.org/wiki/Indiana_Pi_Bill
> The Indiana Pi Bill is the popular name for bill #246 of the 1897 sitting of the Indiana General Assembly, one of the most notorious attempts to establish mathematical truth by legislative fiat. Despite its name, the main result claimed by the bill is a method to square the circle, rather than to establish a certain value for the mathematical constant π, the ratio of the circumference of a circle to its diameter. The bill, written by the crank Edward J. Goodwin, does imply various incorrect values of π, such as 3.2.[1] The bill never became law [...]
Generic arguments about personality and character don't invalidate specific arguments about IQ.
If I indulge your "argument", from what I see on twitter is threads full of very openly nordic-/white- supremacist keyboard warriors and zerohedge trolls. Of all the people to call out in these threads, Taleb seems the most level-headed.
So he looks at the quantitative claims and they are just flat out wrong.
However, the non-quantitative "analysis" is just as faulty.
First huge mistake is that he claims that IQ correlating better with outcomes at the low end somehow shows that it doesn't measure what it claims to measure, "raw ability". That is nonsense, because ability is just one factor, but a necessary one, for outcomes. So if you don't have the ability, you're not going to get the outcome. But if you do have the ability, that doesn't mean you're going to get the outcome. You might not be motivated, for example.
In fact, I remember a talk from a couple of years back showing that financial success tends to be anti-correlated with the highest IQs, because those people figure out that it just isn't worth it.
But anyway, the correlation being stronger inn the "non-ability" part of the distribution does not show that it's not measuring ability.
Another whopper is that he claims that performance is is fat-tailed, without citing any evidence. According to Albert-László Barabási[1] who studies performance, success and how they relate, performance is more closely normally distributed, it is success that is fat-tailed.
And of course Taleb doesn't seem to distinguish between performance and success, which makes the whole rant pretty worthless.
https://hypebae.com/2018/11/how-to-wear-birkenstock-sandals-...
And frankly, if there is any doubt that IQ maps to intelligence, then its abuse over the last half-century has done immesurable harm to individuals and given considerable fuel to racists.
IQ tests turned out to be pretty useful when investigating the effects of Lead in paint/gas/industry/other. See Herbert Needleman's research from the early 80's.
The test itself isn't really good or bad (bearing in mind most modern iq tests are heavily scrutinized for bias in race/class/income/location/etc) it's just a tool.
Taleb argues precisely that IQ is only a good measure of unintelligence, not intelligence, and that this prevent IQ from being meaningful. He showed that if you IQ test 10K people, 2K are dead (0 IQ) and then test their performance on something, you'll still end up with correlations that'll get you published in prestigious journals, and peddle nonsense conclusions. https://twitter.com/nntaleb/status/1115193783145123840
(the point being that in that scenario, 0 IQ perfectly correlated with 0 performance, can make the population correlation look significant even when "alive" IQ is completely uncorrelated with performance, which, in that example, it is)
Taleb's argumentation relies on fictional scenarios whose conclusion do not match what he would have gotten looking at raw data instead.
Caloric intake in children correlates to height very well below the median(+), and then it stops. You can't grow arbitarily tall by eating more food, but if you underfeed a child enough, it will end up shorter than its more-fed siblings.
In case you need some justificaiton for this intuitive claim, here's some tangential data: https://www.scientificamerican.com/article/how-much-of-human...
" In contrast, in developing countries, nutrition deficits lead to a lower heritability. The fact that the mean height of the U.S. population has almost plateaued in the past decade suggests that the nutrient environment has almost maximized the genetic potential of height, at least in this country. Improved nutrition elsewhere may have similar benefits in terms of stature."
(+) I use median here to mean some unspeicified percentile, as you did.
Here is a good explanation: https://twitter.com/sean_a_mcclure/status/115548736901923225...
But you brought up the same point as the parent. 0 IQ correlates perfectly with death. Really low IQs correlate perfectly with mental disabilities. High IQs correlate (almost) perfectly with socio-economic status.
https://www.ncbi.nlm.nih.gov/pubmed/8162884 https://ehp.niehs.nih.gov/doi/full/10.1289/ehp.7688
[1]: https://journals.sagepub.com/doi/full/10.1177/17456916156099...
[2]: https://osf.io/ezcuj/wiki/home/
[3]: https://science.sciencemag.org/content/349/6251/aac4716.full...
You should have very little confidence in the predictive power of most psychology findings since the founding of the field.
If someone says that IQ is unreliable, they're likely right just given the track record of that field.
can you identify any specific problems with the analysis in that post?
discussions around taleb often have a very cult like aspect to them - a refusal to discuss specific empirical claims made by Taleb and an assumption that the critics are too dumb to understand taleb's deep mathematical criticisms.
1. Crystallized intelligence - essentially your knowledge / what you have memorized (including useful heuristics, experience, etc). This can grow throughout your lifetime.
2. Fluid intelligence - in reality, this is basically defined as your "working memory", which is how many pieces of distinct information you can maintain in short term memory and work with. The magic number is 7(+-2). Anyway, this influences your ability to learn, since if you keep in mind what you just read/heard, you can connect the dots.
Pieces to memorize can be grouped to make them easier to work with.
As a result, an expert in a field will have pieces grouped and organized in long term memory, and WILL appear more intelligent than an amateur in the field, when discussing that field.
In my, subjective, experience this can be enough to work in one field.
When you want to be a Renaissance man, or compete at an extremely high level, low IQ chokes you. Even a difference of 10 points between 120 and 130 chokes you.
Hopefully this wasn't a rant and you know what I mean.
Do they? Where are you getting that from?
>and obviously the IQ test would further serve to protect the power structure of the people who designed those tests.
Not so obviously. No matter what IQ is, it was neither designed nor implemented as a way to 'protect the power structure'. This idiotic power hierarchy metaphor is completely overused by certain ideological sectors. Not everything is best understood as power hierarchy. Not everything is a power hierarchy. When you claim everything is a power hierarchy it is a cognitive crutch not unlike conspiracy theorists seeing the work of the illuminati behind every socio-political event.
It's either a tautology or follows logically take your pick, a system is a set of things working together as parts of a mechanism or an interconnecting network. System integrity is the state of a system where it is performing its intended functions without being degraded or impaired by changes or disruptions in its internal or external environments.
Systems that do not protect their integrity cease to exist or they change in unpredictable ways and become uncontrollable.
> it was neither designed nor implemented as a way to 'protect the power structure'.
It is designed by someone to figure out if someone else is worthy in a certain way. That in itself alludes to some sort of power structure. Also the conspiracy theory libel is a logical fallacy.
OK, let's go with a tautology, in which case it is a meaningless red herring to your argument of power hierarchies. If every system protects its integrity (even those not based on these mythical power hierarchies), then it doesn't really introduce anything interesting to your argument.
Truthfully however 'the System' you're referring to is ill defined. It could mean anything.
>It is designed by someone to figure out if someone else is worthy in a certain way
No. It was designed to identify children who are struggling in school. That was the original goal. That is not the same thing as 'figuring out if someone else is worthy', though I'm sure you can contort the language to fit this. It has since been used in a wide variety of contexts, all of which are varied and differ in the target of study and purpose of study.
>That in itself alludes to some sort of power structure.
No. It doesn't. Just because you say it, doesn't make it so.
>Also the conspiracy theory libel is a logical fallacy.
You're demonstrating the same kind of - if all you have a is a hammer, everything is a nail - type of thinking. You've alluded to the fact that everything can be distilled down to some power hierarchy. That's a tell-tale sign you've come down the ideological rabbit hole.
Here are other ideological cognitive crunches:
- Libertarians see everything as a struggle between collectivism and individualism.
- Communists see everything as a struggle between bourgeoisie and the proletariat.
- Feminists see everything as a struggle against patriarchy.
- Conspiracy theorists see everything as a struggle against the Illuminati (or some shadowy cabal)
The world is complicated. Ideology helps to make sense of it by introducing useful abstractions ... but only to a point. As much as ideology helps, it can also be a crutch.
Technically, all logical implications are tautologies. Logic's weird like that.
"No. It was designed to identify children who are struggling in school. That was the original goal."
Yes. And as long as IQ testing is used to identify disabilities of individuals, I doubt seriously that anyone will find anything controversial.
On the other hand, one of the more immediate next uses was to argue for limiting immigration to the United States (from, IIRC, southern and eastern Europe; I think the drunken, sex-crazed, dim-witted Irish thing had blown over by then).
Sure. That's why we tend to argue by induction ... even when we claim deductive reasoning. Deductive arguments aren't very interesting.
To give them credit, they worked doubly hard when they did show up and were sober and usually got 90% of what they were supposed to clean for the day done. Whatever other places they were trashing in IV on their nights out I can’t speak to
I think trying to debunk the current utility of IQ by invoking its original goal is almost a sleight of hand. Unfortunately it didn't stick to its original goal. IQ tests have been administered to people of all ages for all sorts of reasons. How much has it deviated from its original goal? And more importantly, how do we restore it?
I wasn't trying to pull a fast one against anyone. The specific line was a response to OP who stated: "It (IQ) is designed by someone to figure out if someone else is worthy in a certain way". I meant to convey that IQ was neither designed initially for that purpose, nor is it solely used for this purpose today. The metric is used in all kinds of studies for all kinds of reasons, and yes, it probably is used in admissions as well.
By the way, I wasn't making a case for the utility of IQ at all. I just hated OPs characterization of IQ being a nefarious tool of some power hierarchy. That is not the right way to look at it.
In human societies and in liberal democracies there are always power structures, those structures are not necessarily power structures of a certain ideology or a group because as you said ideologies are abstractions... and individual belief is always synthetic and syncretic ... however that does not mean that power structures don't exist.
Where IQ is often used as a tool to identify "gifted" students, it's rhetorically written off as a tool to help those in need.
> You've alluded to the fact that everything can be distilled down to some power hierarchy. That's a tell-tale sign you've come down the ideological rabbit hole.
No I have not, you are guilty of the exact same sort of thinking you are accusing me of instead you are preferring the status quo narrative which is that it's all milk and honey which it is not. In the end we have systems that people use to govern and perhaps they have some tolerance and fluidity but that does not mean that as an abstraction they do not exist as a certain set of ideas that are carried by a set of individuals who seek to protect those ideas and the power structures that go with those ideas... because their power depends on the preservation of those ideas and those systems.
IQ is a part of that equation in that you only let intelligent people in your country and then you school them to determine who useful they are you to your system and you only allow those who are intelligent enough and predictable enough to have power in your system. Thereby you protect your power and your ensure the integrity of your system.
https://medium.com/incerto/iq-is-largely-a-pseudoscientific-...
He's showing us that psychologists' statistical methods are faulty, that IQ advocates don't understand the underlying mathematics and blindly plug things into R. Lo and behold, in rebuttal, psychologists try to prove Taleb wrong by plugging more things into R.