NAP allows for "retaliatory violence" (had aggression but as pointed out below is not the correct usage of the word aggression).
One of the difficulties, as you point out, comes when people don't realize that they are aggressing others. An example being when they attempt to limit who can get married to who (whom?).
Again, there are good intentions behind such aggression (limit the right to use marijuana to protect children) but the results of aggression seem to always lead to more aggression (creation of an illegal drug black market leading to instability in Mexico).
Property can never be theft as theft only exists if property exist. You can't steal something if no one can "own" anything.
> what gives you the right to compel me to defend your property?
No one has a right to compel you to do anything.
However, you will act in your best interest. How about I pay you to defend my property?
Or, perhaps it is in both of our interests if we defend each others property. After all, if you let others steal my property, yours will probably be next.
This holds even more true for rights.
The intentions of laws with compulsory fees (taxation) and compulsory who marries who are based on good intentions (in the minds of those who create them).
In both cases, these good intentions are based on theft of basic rights. But, in the long term, how can you make society better off when your core beliefs are based on theft? ("Unlike both parties' ideologies, which are, indeed, crafted out of many different unrelated parts serving special interests.")
The "batshit crazy" ideals of libertarians is to base your good intentions on non-aggression: not theft.