The problem is that what is considered "hate speech" and "misinformation" is often very subjective. How would someone even "invade" other people's communities? Everyone chooses their own timeline after all.
If all you ever do is read and never post anything people might interact with, then that's true. That's not much of a "community" though if nobody ever posts.
Groups that promote hate speech and misinformation do so by invading other communities and being disruptive. Brigading and doxxing have been their tools for a long time.
Personally, I've seen that breaking up groups that promote hate speech and misinformation helps the remaining members to enjoy the community in peace.