On another note, I also get the impression that you write to a certain type of person. Possibly a type that is similar to you. People like Jeff who take issue with your essays are simply not the target audience (whatever that means). As someone who is currently caged in a 9-5 job with a big company, I have to say that your essay resonated deeply with me...and I loved it. I appreciate the way your essays frequently force me to take a good hard look at what I'm doing, where I want to go, and what I need to do to get there.
Keep in mind that different situations demand different writing styles. For instance, if you are trying to resolve a bitter conflict, "writing things that cannot be disputed" is just the way to go. In that situation, your #1 objective would be to find things that both parties agree on, and build up from that. (This also might be a good way to win an argument.)
There's a really dense book called "Difficult Conversations" (Stone, Patton, Heen, Fisher) that covers, among other things, how people frame discussions in terms of their identity. I think if I had adsorbed that book better I would have predicted that this essay would deeply offend some people.
One other reason is that you have a touchy subject. Tell anyone they're doing the wrong thing (implicitly or unintentionally) and you'll get the same response. People are very sensitive about their own choices (stubborn), especially ones that are hard to defend rationally (religion, major, job, etc).
I picked up this habit in the Harry Potter fandom, which aside from being a community of writers also happens to be 99% female. Women's writing tends to include many more phatic expressions, because (particularly in something as socially-constructed as fandom) its purpose is often relationship-building. I adapted to fit in; before then, my writing style often tended to be brusque, mechanical, and to-the-point.
I still use my old style in technical posts, where I figure the reader can deal with any unintentional rudeness. For example, compare the last paragraph of this: http://news.ycombinator.com/item?id=144001 to this comment: http://arclanguage.org/item?id=3103
I agree, but I don't think it's because they're hard to defend rationally. Instead, I think it's because they're an essential part of their identity. If I criticize you're choice of colors, that's not something that you can defend rationally, but you won't be too concerned if you're not much of a design person. On the other hand, if you're an academic who built a career on some theory, criticism to your theory will be very personal. The issue here is one of cognitive dissonance. It's very difficult to differentiate between critcism for someone's choice and general mean-spirited feeling for that person.
I'm always surprised by how offended people get by things I write. It seems totally unpredictable. I didn't expect people to be so offended by this one. In fact, I thought I was saying something rather smarmily ingratiating, if anything: that the famous startup founders you keep reading about in the press are not that different from you, but that they just have, in effect, a healthier work environment. See the last paragraph.
And yet somehow that message has gotten completely twisted around. It's as if people wanted to misunderstand this essay.
I've been mulling over why this happens, and one reason is certainly the one you suggest. I try to cut every unnecessary word, and I don't say things unless I'm pretty sure of them. The result sounds arrogant, because it doesn't have any of the hedging people usually surround ideas with to make them palatable.
But there's no alternative. People won't read essays if they're too long. If you want to get a lot of ideas into an essay short enough to read, you have to be so curt you sound arrogant.