> Why is voluntary segregation a bad thing?

Because everyone who doesn't like the consequences of their decisions will blame everyone else. They might even be right, but it could/would increase conflict. Boundaries are frequently decided via violence.