That depends. Distance from somebody you really really dislike can make it easier to work with that person. What you dont see can not annoy you and you can limit communication to written.