How many people commit violence, and how many people are victims of violence, are two very different things. You could live in a society where only 1% of people commit violence, and yet the remaining 99% are living in fear, because each of them was repeatedly a victim of violence.
But if you have 1% of people ready to initiate violence, and let's say 3% of people willing to use violence in self-defense, suddenly life becomes much safer for you, even if you are among the remaining 96%. Not because the bad guys would hesitate to hurt you, but because they are likely get in trouble before they get to you.
People often confuse these two numbers. For example, they look at some statistics and think "20% of women report having been victims of domestic violence... oh, that means that 20% of men must be violent abusers", and they don't realize that the statistics also include some violent men who abused five or more partners each, so the actual number is probably much smaller than the 20%.
Probably not. The reason we're not permanently locked in a life or death battle against each other is that very few humans like committing violence. It's a pretty terrifying view of the world to think that all that's preventing someone from perpetrating a home invasion on you is the threat of violence.