Well that only works with people that are not afraid of fighting you and possibly winning against you - and there's always somebody that's better trained and/or stronger then you. When you encounter that kind of person, if you challenge them, they will step up and fight you, because they will feel they have no other choice. Reasons could range from pride, (imagined) loss of (self) respect to just being aggressive bullies that enjoy hurting other people. Of course if you're 6'7'' and 240 lbs not a lot of people would want to start arguing with you in the first place.