One of them is that this "men with guns" phrase is often deployed to plant some kind of "government over-reach" meme around the most trivial of adult transactions, like paying parking fines and property taxes.
It betrays a childish political naivete. The reason I say it's childish is that it's egocentric, as if the concerns of the speaker (to park where they want, to own large plots of land, that abut other people's land, without any consequence or restraint) were the only ones in existence.
That house is served by roads, protected by police and fire services, and must be in compliance with various perfectly reasonable regulations (like brush clearance). That costs money.
And last, you know where your phrase, "All power comes from the barrel of a gun", comes from, right? Good old chairman Mao, one of the top guys in the 20th century rogues gallery. In a democracy, with functioning courts, this saying is more hyperbole.
I think a more relevant notion is that the state is the entity that has a monopoly on violence (due to Max Weber, see http://en.wikipedia.org/wiki/Monopoly_on_violence).
That's a ridiculous simplification. You may as well say, "all power comes from tanks," or, "all power comes from nuclear arms."
If the county takes your house, the sheriff will come out to escort you off the property. Guns don't come into play unless you pose a threat to their lives. Even if guns do come into play, that's still not ultimately the force that moves you. Being outnumbered by a better organized group is what ultimately forces you out. The guns are just one small part of the puzzle.
Water, electricity, and gas infrastructure can be paid for by taxes on those services, and people who don't want them can not purchase water, electricity, or gas from the "grid". Roads can be paid for with gas taxes, and that way are paid for proportionally by those who use them.
And to add to the GP's post, government can choose to seize your land at any time using eminent domain laws are are more and more abused every day. So, whether or not you agree with Libertarianism, I do think he has a point that there are a lot of similarities between buying and renting that a lot of people maybe don't realize.
I think that the choice of paying property tax and living on an island is a false dichotomy. We could theoretically live in a society without paying property tax, where the government can't seize your land.