Because business decisions should be made based upon facts and data, not based upon hunches and gut feelings. Going with your gut basically means throwing out the data and letting all of your personal biases decide the matter, consciously or unconsciously.
If it's a question of being able to work with others, then you can develop principles and behavioral interview questions around what you want your coworkers to be like. That's a bit of a pain in the ass, but empiricism was always more of a chore than just following your gut instinct.
I feel like it might be almost impossible to remove gut feeling from the equation alogether, though, since a lot of the problems that wreck teams really do come down to "X really doesn't like Y's personality". At some level, isn't a negative gut feeling expressed by one of your team members as valid a data point as you can get to predict how well the person will work out on that team?
I'd never let a good culture fit override a poor technical interview, but I've seen the consequences of hiring a brilliant programmer that happens to be an asshole, and it's not pretty. Am I misinterpreting what you're saying, or do you really think social skills should be entirely ignored when it comes to hiring?