I think there's a difference in feeling bad for someone with good intentions who got unlucky (vaccine adverse effect) and someone who active tried to make to world a worse place, fucked around and found out (antivaxxe).

This whole pandemic made me realize that Paradox of Tolerance is much more general - there is also Paradox of Empathy and Paradox of Kindness and bunch others.