Anarchism without pacifism is a contradiction, and pure anarchism can only succeed as far as can pure pacifism. Neither are likely to happen, so again, they are more ideals that should influence a more virtuous way of living, and perhaps governing, than near goals to be achieved by force or distance. As such, it's pretty obvious that "driving them out" is as far from virtue or pacifism as you can get, and is totally counter to the spirit of volunteerism. Neither imprisonment nor exile are tolerable solutions for the pacifist.