I am not a doctor, but my understanding is there is a small bug significant number of people who cannot get certain vaccines due to e.g. immune system issues. "Herd Immunity" through a highly-immunized population generally protects these people because there are fewer reservoirs for a given disease.

Edit: The reason people are mad at anti-vaxxers: their foolishness directly impacts the welfare of some already unfortunate folks.