If my rights are granted by God (and included in those rights is the right to private ownership) then if someone invades my land they are violating God's law.
Private property is the basic requirement of wealth. It is defined by law to codify what existed for the protection and benefit of everyone. I'm basically quoting Aristotle here.
Rights aren't granted by men, they're asserted by men. We have the right to free speech because we assert that we have the right to free speech, and the collective will of those who wish to assert the right to free speech is (currently, in this country, for the most part) greater than the will of those who wish to restrict it. This is why it's so important that we all agree on what rights we do have, and that we all spend a lot of time vigourously asserting them.
Without a basis for defining rights you're leaving room for those rights to be trampled on by the next popular tyrant who comes along. That's what I'm concerned about.
In a sense, "property" does not exist. Property only exists because laws exist to protect it. If the country you live in is invaded by a more powerful nation, how much do you think your "property" matters at that time? It's just a set of limited rights granted to you by the government, not unlimited license to do whatever you wish.