Your approach may have been more cautious but out of principle, I probably would have asked him to release his grip, followed by shoving him away if necessary. Normalizing those kinds of intimidation tactics is not in the interest of anyone. If more individuals were to stand up for themselves, it's likely that the person in question would reconsider employing such tactics.