Consequently, the ones who are left out of the resulting parties are nearly by definition outliers. If one studies statistics, one of the things you'll learn is that in anything that has even a faint resemblance to a Gaussian distribution, the outliers are always more diverse than the main body. So on those fringes, you can always find some real nutcakes, along with a group of people whom you may merely find eccentric but forever bear the burden of being tarred by association with the loonies.
(It isn't necessarily the best plan to judge a fringe belief by an even more fringe set of loonies; for every stone you might be inclined to toss at the "other side", it's worth bearing in mind you've got a matched loonie on your side. Naturally, the loonies are the loudest, everywhere.)
I claim to be libertarian, but probably even more accurate would be call me a Classical Liberal - http://en.wikipedia.org/wiki/Classical_liberalism (Albeit not every bit of that; 18th century Classical Liberalism can not be simply blindly applied today, but I think it's closer than later philosophies, or, perhaps, ironically more appropriate for the 21st century than the philosophies of the 20th, which I think were too greatly influenced by the accidents of the Industrial Revolution, which fade as we continue through the Information Revolution.) Anyways, following through that on Wikipedia would probably give a helpful perspective on the general ideas, moreso than a loud loonie. There is an actual respectable intellectual tradition here, even if it is not the one currently in favor.
> The use of the word libertarian to describe a new set of political positions has been traced to the French cognate, libertaire, coined in a scathing letter French libertarian communist Joseph Déjacque wrote to mutualist Pierre-Joseph Proudhon in 1857, castigating him for his sexist political views.
Just my two commie copeikas...