Depends on the threat you're facing.

If you're worried about NATO or former Warsaw Pact aggressors, no, men with rifles are not a deterrent.

If you're worried about a neighboring region's warlords sending people to steal your food, kill the men, rape the women and conscript the children, yes, men with rifles can convince them that their activities will be more fruitful somewhere else.

For a threat like Russia or the USA, your only recourse is to ally with one or the other.