There are plenty of brilliant scientists who are held back by their excessive rationality, or whatever you want to call it. Another great book on this topic as it relates to econometrics (as opposed to physics) is "The Romantic Economist" by Nicolson.
Sociology is very important to study. The tools we have for it are not great, in various ways. For example, one can study people and societies by reading Balzac's writing, but the number of people who can produce that sort of thing is fairly limited. We can try to do controlled experiments, but the way we do it in practice is not great. We really really need better tools here...
So yeah, personally, I have utmost respect for the complexities involved in sociology - while at the same time I absolutely hate all the bullshit that's being done because doing actual research feels too hard.
There is no stereotypical "scientists' disdain" for those topics. It is vital to understand history, social sciences, economics, politics, and the arts. What is disdained is the way these topics are, in the present world, not pursued along more rigorous lines.
Too many people like to look good by producing grand-sounding theories about these topics, without putting their ideas through a more rigorous process to check whether they are actually true or not. Then we get a cohort of followers who believe these theories as if they were "as true" as mathematical theorems, but they are not. This wastes everyone's time, and worse.
One can certainly picture an overly-specialised scientist who doesn't know anything other than the specific field they chose - I guess this is what you mean by "excessive rationality" - but I don't see many cases of this actually existing. What I see in far greater numbers is the problem I mentioned in the previous few paragraphs, as well as people using the "excessively rational" image as a straw-man argument to attack scientists, to distract the world from their own flaws, of "not enough rationality".
However, it's too easy for bullshit-artists to hide behind this, as a way to deflect criticism of the theories that they're proposing, or the general area of work that they've chosen to pursue.
We're all human so it's perfectly reasonable to believe in something that's unproven - but then be honest about it, and explore the topic from a critical viewpoint without getting personally defensive. Unfortunately I see many more cases of this, than instances of the fallacy-fallacy.
Edit: maybe he was a nihilist, I'd tolerate nil as the only non quantitative number. Sorry for the rant, but that's what you get for anecdotal evidence.
Feynman certainly did.
It's not obvious to me that measuring scientists or research quality has anything to do with the "spirit of science." Just because science involves very carefully measuring the objects of study, it doesn't follow that science is best served by trying to meta-measure that process. And there is evidence that it can be harmful.
And pandemonium models [1] of computation show that the right rules of cummunication can produce astonishing order from chatic self interest. If scientst were forced to pre publish their methods and share all data -- science would probably work better. This is true even if this change had no effect on their behavior, or even if they made every effort to game the system.
This imrpovement to science doesn't ask a committee to define or measure anything ineffable, and it doesn't expect individuals to change thier behavior. It just changes the rule of interaction in a way that better favors the systems epistomological progress.
Of course the more noise you have, the less efficient the system is, so it's good to incentivize people to do honest, objective work - thus pre-publishing / pre-registering, sharing data and algorithms, etc. are all good and important goals. But so should be changing the metric affecting the aggregate - like making sure scientists are actually incentivized to replicate previous work.
People just don't like this real solution because it means many (really, most) of the wild speculations they hold as dear dogma (and have been calling science) will need to be reassessed using the correct metric. There will undoubtedly be a lot of egg on many faces.
Our need for "objectivity" here is to a large extent driven by the centralization of the funding allocation that http://mcadams.posc.mu.edu/ike.htm warned about and has since come to pass. If funding were more decentralized, lack of objectivity in any particular funding source would be _much_ less of a problem.
There's lots more in there that usually doesn't get quoted (unlike the bit about the military-industrial complex) and is spot-on prescient.
We follow these principles out of observing history. But part of it is an act of faith, I'll give you that. What sort of "evidence" are you referring to, that it would be harmful?
A problem is that any such metric is only ever going to be a proxy for some assumed real value, and such a proxy is always going to be controversial, particularly if it's used to control access to funding.
Which I think at the end of the day is the big issue here. All this soul-searching is just a big attempt to decide what we spend money on. Who gets to be a scientist, and who gets to become a person with a lot of useless knowledge and an incurable sense of failure.
And who goes into finance, software engineering, or some other tangentially related field where the skillset is very useful.
Certainly we need to look at our social reward/incentive systems from an adversarial point of view, in science, economics, politics and elsewhere. Game theory is useful for that, and it can help us develop systems that are less-easily susceptible to the flaws that the article mentions.
Academic scientific activity is still mostly a social activity because it is done through the communities and the social interaction between scientists and institutions. Specific problems may be solved by specific brains or groups of brains, but the output and the collaboration is all social. The rewards and incentives are all social systems. And at the end of the day, any time we write a blog or post a comment, it's all social. This is social.
But the disdain for social sciences by non-social scientists is discrimination only warranted to the extent they can avoid social and professional interactions with them, and feel immune from any criticism for their bigotry. The moment a scientist wishes to study the social aspects of science itself, they have no choice but to accept social science as a science.
But this is okay because reputation has no consequence in problem solving, because reputation is a social device, and not a solvent. The track record of a field, a department of an institution, the history of publications, or the publications of an individual social scientist are all largely irrelevant when faced with your own research targets. The only thing relevant is the available data and the premises chosen for any model. There is no need to criticize the scientific integrity of the work of others if you can do better. Do it yourself if you have to.
Actually I thought yesterday about how to misconstruct a human detector for photos, one possibility would be to have face detection and then measure the average pixel color of the face. That would get me probably quite nice detection rates if I test on a dataset that has a racial markup as the average CS lecture. The thing here is, that I push the false negatives purposely to people of color. It is not hard then to invent a story about how 'mathematics proves' that black people are more similar to apes than white people.
We currently just don't have anything to ground models of human interaction in,^1 and what is worse most people treat models just as previous generations treated prophecies. They don't understand math and believe in it because they don't understand it.
^1 I am actually not saying that models are worthless, I am saying that most complex models only show what the author wants them to show and they can be as easily manipulated as an essay.
Someone goes and builds a bridge. The bridge collapses because they applied mathematics in a way to maximise profit, with a safety margin just beyond what they can get away with by existing regulations. 5 million other bridge engineers also do the same, so that eventually bridge engineers get a bad rep, and people think bridges are awful awful things. This doesn't mean mathematics or quantitative analysis is at fault - and in fact these tools can be used to examine the incentives and other social dynamics that led to these situations, to be able to fix them more effectively in the future.
no, not of gravity or dark matter.
As for dark matter, the reason we don't have a good idea what dark matter is, is that is has almost no effect on normal matter. So the reason why we can ignore dark matter everywhere except at the frontiers of physics is, that dark matter only matters when one spends awesome resources to try to detect it.
No, you can't. What an outrageous claim to make. Prove it constructively, preferably.
I disagree, because of https://en.wikipedia.org/wiki/Goodhart%27s_law
The problem is that we want "quality science" (whatever that means!), but we don't know how to quantify that, or indeed really define it. So a quantitative metric will necessarily be measuring some sort of proxy or set of proxies for "quality", and then you will get people optimizing those proxies, not "quality". To the extent that those proxies miss something important, it will be underinvested in.
Unfortunately, I don't have a better proposal, perhaps short of taking the warnings in http://mcadams.posc.mu.edu/ike.htm to heart (the ones that are NOT about the military-industrial complex). Doing that might change the general funding climate sufficiently that the need for deciding "quality" like we do now may simply become less critical.
I've run into that myself, as described here: https://news.ycombinator.com/item?id=12271097
Mechanism design (reverse game theory) is subfield of game theory deals with these issues.
Edit. Prof. Google to the rescue [1].
They need something like a "Journal of Meta-Research", which would research the best ways to perform research. You'd think something like that would get a lot of funding...
The reason why scientists have a natural "disdain" for sociological studies, is because they don't have these qualities. Many of the arguments aren't convincing from a critical angle, and only convincing to people who have a pre-existing bias towards certain conclusions. Yet the scientific culture of today is falling into that trap itself.
The article criticises metric-driven incentives, but it is not metrics (the general concept) that are at fault. It is the choice of metric, and the meta-analysis of this that is lacking. These choices are themselves often backed up by non-scientific vague arguments, of the similar sort that scientists often criticise other fields as depending upon.
We must certainly not conclude from these studies that quantitative analysis is itself what is at fault. I know the article doesn't explicitly say this, but it hints at it - using suggestive phrasing like "disdained sociological studies" and referring to all "metric incentives" as a single group - and it is a point I have seen made by many non-scientists. That is, using these flaws as a straw man to attack the very qualities of what has made science so successful and useful.
To improve the situation, we must reject these straw-man arguments against science, and develop better methods that are more quantitive, over a broader spectrum of what is being analysed, and that are more self-critical.