Anarchism without pacifism is a contradiction, and pure anarchism can only succeed as far as can pure pacifism. Neither are likely to happen, so again, they are more ideals that should influence a more virtuous way of living, and perhaps governing, than near goals to be achieved by force or distance. As such, it's pretty obvious that "driving them out" is as far from virtue or pacifism as you can get, and is totally counter to the spirit of volunteerism. Neither imprisonment nor exile are tolerable solutions for the pacifist.
> Anarchism without pacifism is a contradiction
Please expound on this - that statement sounds unintuitive and generally the opposite of what I'd expect
Anarchism rejects coercion. Pacifism rejects violence, including coercion. They go hand-in-hand. Why would you expect anarchism to be the opposite of pacifism?
Probably because its easy to conflate Anarchism, a clear ideology, with anarchy, a state of social/political/governmental organization that may coincide with Anarchist ideology, but may also more closely resemble Hobbes' state of nature: "solitary, poor, nasty, brutish, and short."
The basic enforcement mechanism under every a-c system I have ever read is to "drive them out of town." Thieves, rapists, ect will be banished from group. We have only to look to the past to see just horrible of a system that can be for people who have a different skin color, sexual orientation, ect from the dominant group.