I think we forget that we're dealing with brilliant, exceptional people when we speak of these authors. Like many of their characters, they're far beyond the norm in human intellect and capability. These are by far my favorite quotes of Heinlein and yeah, show that he demanded personal accountability and liberty, and probably didn't like unexceptional people. He loved brilliant and independent people, and characters, and was an asshole to the rest, and to those who opposed his philosophies.

“An armed society is a polite society” is one that's floated often today among those pro-gun.

“A human being should be able to change a diaper, plan an invasion, butcher a hog, conn a ship, design a building, write a sonnet, balance accounts, build a wall, set a bone, comfort the dying, take orders, give orders, cooperate, act alone, solve equations, analyze a new problem, pitch manure, program a computer, cook a tasty meal, fight efficiently, die gallantly. Specialization is for insects.”

I've encountered people that take these as him being a fascist, asshole or prick. Personally, I agree with him on both counts, and consider most people to be lack contributions to the human race for their lack of desire to learn and become competent, or for their inability to learn. If that makes someone an asshole, so be it.

You have to be careful with quotations that come from his fiction (as I think both those do) as opposed to essays or letters. That said, Heinlein did love writing the "competent man" character so there is a strong argument that this reflects his own views.