Action against those in positions of power who cause harm to others is meaningfully different to action against those who are already victimised. The first is a movement for social change that increases the net good in the world, the second is simple oppression.

In the same sense, one might say that overthrowing a fascist dictator is more morally justified than overthrowing a peaceful democracy to install said dictatorship. Both are violence against a government, but they exist in different moral contexts.