> He's not a social scientist, and I don't think he'd appreciate an anthropologist evaluating the state of affairs in quantum computing research.

But, from things I have read elsewhere, there does seem to be a reasonably convincing body of evidence that there are systematic biases in social sciences. Some of this perhaps stems from discrimination (e.g., in hiring practices) based on political allegiance. Jonathan Haidt (a social scientist) has spoken out on this topic[0], including specifically on sociology[1]. While that issue likely affects academics in general, political leaning likely has a much smaller influence on one's research in physics and chemistry, compared to one's research in sociology. It's true that Aaronson is not a social scientist, but I don't think that means he is wrong.

[0] http://people.stern.nyu.edu/jhaidt/postpartisan.html

[1] http://www.mindingthecampus.org/2016/02/a-conversation-with-...

> there does seem to be a reasonably convincing body of evidence that there is systematic biases in social sciences.

But Aaronson makes a far more specific assertion than "there is evidence for systematic bias in social sciences".

I agree that there is political bias in social science departments. It is painfully clear to anyone who ever sets foot in one, especially as an outsider (much like the sexism in SV). I think that overall it harms research, perhaps significantly, and it must be addressed. But outsiders sometimes view this (rather obvious) criticism as a complete refutation of the field(s), which it is most certainly not. To their credit, social researchers actively study this bias without need for external pressure, and even acknowledge it's a problem, something which cannot be said for other disciplines (one could argue that it is social researchers' job to study precisely such things, but at least you can see that they're not intellectually dishonest enough to avoid the issue or to dismiss it as inconsequential).

BTW, in general, I think that outsiders usually tend to overestimate internal criticisms (in any discipline). When (internal) critics say "our estimate of the variable x to be 10 is far too low!" they really mean "x is more likely to be 12", while outsiders interpret it as "x is probably 1000".

> political leaning likely has a much smaller influence on one's research in physics and chemistry, compared to one's research in sociology

True, but the correlation between political leaning and social research may very well be a reverse causal relationship: that one's research in sociology is far more likely to affect one's politics than research in physics. If there is an over-representation of a particular ideology in the social sciences (although it's hard for me to put all the social sciences in one category, because they are quite different from one another -- more different than, say, physics and chemistry), it may serve as evidence that some common and politically pertinent ideas arise from the research itself[1] as much as evidence of anything else.

BTW, I've read the interview with Haidt that you linked to -- thank you for that -- it is truly horrifying. But I get the sense that this is a rather American phenomenon. I certainly didn't experience anything close to it in magnitude at my university.

[1]: And I don't mean necessarily from the results, but possibly from the process.

There are systematic biases in the social sciences, and there are systematic biases in physics. Read Lee Smolin's The Trouble with Physics. As they are social systems, they will always be biased. And the most dangerous bias is the one that claims itself unbiased, fully objective.

Physics can collect so much data that the generally accepted standard for results is five sigma. There are biases in physics, but it's much harder to delude yourself into a five sigma result than a two sigma one--in the end the right answer becomes obvious.

> Physics can collect so much data that the generally accepted standard for results is five sigma. There are biases in physics, but it's much harder to delude yourself into a five sigma result than a two sigma one--in the end the right answer becomes obvious.

That's in the case of clear-cut predictions from theory or empirical relations. But bias generally arises in a more subtle fashion than statistics on measurements, though. Even if you have a "5-sigma result", it still needs to be interpreted, and that's where biases can rear their heads. On possible manifestation (though not the only), is if one ascribes to a particular theory, one is generally biased into interpreting those results within the framework of that theory. And if one isn't careful, that can lead to diminishing the weight of evidence that conflicts with the theory (e.g., "it's 5-sigma, but those data aren't actually relevant to this issue").

As an example, one area I'm actively researching is the emission from molecules in galaxies. We see that some galaxies show enhancements in emission from a specific molecule (HCN). Everyone agrees that the enhancements exist but there's wide disagreement on what causes it. And much of the discussion around it parallels people's opinions/biases about which is more important for a galaxy: how many new stars it is forming versus how active the supermassive black hole is at the galaxy's center. The preferred interpretations for the extra HCN emission seems to correlate very well with people's prior notions* (which is not surprising). Of course, more data will eventually provide a concrete solution to this question, though what new data are collected is often chosen within an existing framework.

* - I'm sure it's true for me, even though I try to mitigate it as much as possible. But we often have trouble recognizing our own biases.

>That's in the case of clear-cut predictions from theory or empirical relations. But bias generally arises in a more subtle fashion than statistics on measurements, though. Even if you have a "5-sigma result", it still needs to be interpreted, and that's where biases can rear their heads.

I agree, interpretation is the tricky thing. The difference is that in social sciences, both the result and the interpretation are suspect.

> There are systematic biases in the social sciences, and there are systematic biases in physics. Read Lee Smolin's The Trouble with Physics. As they are social systems, they will always be biased. And the most dangerous bias is the one that claims itself unbiased, fully objective.

I never claimed physics was bias-free. I just said that political affiliation likely imparts less bias on physics than it does on sociology. And that was just one example of a possible source of bias.

But that's pretty much what Aaronson is implying. That physics can be cold, calculating, unbiased while the social sciences are the opposite.

There is plenty of dogma in the physics world as well. For example, just look at the way people defend their favorite interpretation of quantum mechanics as absolute truth.

> just look at the way people defend their favorite interpretation of quantum mechanics as absolute truth.

Yeah, but these people are rarely actual physicists. A physicist might discuss "interpretations of quantum mechanics" over lunch with a colleague in a fun, speculative way, but it makes little difference on the research they perform. More importantly, it doesn't affect the predictions that their research makes. This is not the case with the social sciences.

It absolutely does affect the predictions research makes, because funding and PhD students are assigned to whatever models are most popular at the time. For a long time that was string theory, and a lot of departments considered other approaches to be non-optimal for career progression.

Which sounds fine if you believe that heterodox but still valid science gets into the mainstream eventually - like (e.g.) the theory of continental drift did.

But what if useful science falls through the cracks? What if we could have had a valid theory of quantum gravity by now, from a left-field approach that never had funding or PhD students to research it?

How would anyone know?

Are you sure you follow actual physicists? Have you ever heard Sean Carrol talk about how anyone who doesn't believe in the MW interpretation is pretty much stupid?

I mean, yes, there are a few physicists who have turned into science popularizers, but they're no longer at the forefront of scientific research (e.g., Michio Kaku, Neil deGrasse Tyson, Brian Greene).

I do quantum chemistry research. Do you recognize any of the names: Colbeck, Renner, Alavi, Booth, Kohn, Sherrill, Gill, or Pople? Colbeck and Renner are about as philosophical as you're going to get for scientists who currently perform actual research (Tegmark and Baez are in categories of their own).

Do you put Sean Carroll in the same category as the other science popularizers?

Also, isn't quantum chemistry too applied for there to be significant arguments about specific interpretations? I only know QC as it concerns DFT or QMC (I do semiconductor modeling)

No, he still publishes quite a bit. But I'd say it's unusual to find someone who is both a researcher and a popularizer.

> Also, isn't quantum chemistry too applied for there to be significant arguments about specific interpretations?

I suppose that is mainly the point of my original post; the vast majority of researchers who utilize QM do so without regard to the philosophical aspects of the subject. Wouldn't you think Sean Carroll is an outlier in that sense?

But then I could make the same point about the vast majority of humanities academics. Most work isnt paradigm shifting, it's just building on top of currently accepted theories.

And that's my actual point. When it comes to rejecting or accepting new models, there is much dogma in physics as there is in sociology, because the underlying problem is human beings not the subject of study. Even Einstein dogmatically said "God does not play dice", rejecting QM based on his intuition about how the universe works rather than being cold and only looking at observation.

Personally, I don't think there is anything inherently wrong with dogma, it helps your research become an overdamped system otherwise you'd be wasting all your time following ultimately fruitless leads. The important thing is to use this inherent quality of humans to your advantage.

> But that's pretty much what Aaronson is implying. That physics can be cold, calculating, unbiased while the social sciences are the opposite.

Sure, and I agree that it's not necessarily true. I just wanted to point out that I was never claiming physics to be unbiased. :)

So from now on, do we present our credentials and cvs before commenting to any extent in public? No. The whole point about science is that anyone at all should be able to criticize any piece of work on the basis of the facts.

Let's suppose the criticism is rubbish. All to the good. That will emerge and so will salient criticism, if it exists, that may have escaped the 'credentialized' scientists.

Only my impression of course but reading a few entries in Shtetl-Optimized, I would have thought that Aaronson would certainly appreciate and argue for or against any pertinent comments that some unknown Joe Doe might offer.

All I meant is that there's little point in presuming the statement is correct and then asking a followup, when there's little reason to believe that it is in the first place. Aaronson doesn't support this statement, nor does he argue it -- he just asserts it. In any event, my point had little to do with credentials and much to do with knowledge. I don't think Aaronson claims he has too much knowledge in the social sciences, credentialed or otherwise. Would you also presume his medical advice to be correct?

This is argument from authority. But, in a world where we don't go through people's statements and rigorously examine the provenience of each of them, argument from authority is a valid heuristic.

Perhaps it is an argument from meta-authority, or a meta-argument from authority :) But I don't think it is. The argument is simply that there's little reason to accept a factual statement as correct if it is not supported by factual study.

The reason I'm so sensitive about this issue in particular is that I've personally been involved both with the social sciences and STEM, and I'm extremely annoyed to see people in the latter group put so much emphasis on evidence, and at the same time display such a broad dismissal of the former group without so much as a rudimentary understanding of the field(s). You hardly see historians dismiss research in solid-state physics as you see computer scientists dismiss research in sociology (which goes above and beyond the well-accepted inaccuracy of the latter due to intractability).