But it's actually the left-right axis that's the egregious oversimplification, and as long as people insist on reducing political questions to that one axis, the "discourse" will continue to be as meaningless and "more heat than light" as ever.
That's not what I observer from social conservatives when it comes to their beliefs. The majority want to use government to force their values on society, like having prayer in public school, fighting the drug war or banning abortion.
Also a majority tend to be rather hawkish and in favor of intervention abroad. Libertarians are in favor of none of those things.
I would argue that social conservatives, when pressed on it, only want the government to enforce their point of view because they are otherwise impotent. If you gave them a choice, "Free public school for all, but no prayer in school" or "No free public school, and all the schools are run by your church" their hearts would favor the latter, even if they say aloud the former.
It's worth pointing out that 'the discriminated' refers to both the hated and the hateful. And in fact, the hateful are hated themselves for being hateful.
The law generally doesn't change on an issue until there's at least a sizable minority of people in favor of the change. So when the law does change, it's more of a lagging indicator, anyways.
How can this be? Well, the answer is that while social conservatives sound very authoritarian- and they are- when it comes to things like gay marriage, sex, etc, the authoritarian force for them doesn't come from the State. Ideally, it comes from society and the church. They would rather the government not get involved at all, private beating do the work.
Socially liberal people, however, want the government to protect this behavior. Notice that the Left didn't take the 2015 SC decision and say, "Yay, we've got marriage, we're done." No, they wanted the government to protect the employment rights and other various rights of gays.
Where do libertarians stand? Well, they don't want the government to help! When given a hateful baker, they say, "Hate and bake, I don't care, the market will handle it!" On the spectrum of answers to "How much do you think the government should be involved in the issues?" this is obviously a right-leaning answer. The distinguishing factor is that libertarians don't have any personal gripe against the discriminated, they just think the government shouldn't raise a hand to help them.
tl;dr: The Left-Right access is about government involvement, and libertarians want less involvement, just like their more "socially conservative" counterparts.