Usually whenever I see people evoke the paradox of tolerance they draw more attention to the authority of Popper compared to the nature of it being a paradox. It defers the question of what is tolerant, which depends on perspective as you rightly point out.