Yeah, I think you're right that a big chunk of this is vibes-based in-group signalling. One doesn't usually wind up far out on the political poles if one's generally happy with the decisions and worldview of the experts/authorities.