Content does matter, in an ideal world. But in reality that type of "consideration" will be weaponised against minorities. You are arguing against yourself when you say that we just have to have "due consideration and a good decision making process" to decide what would be allowed. When said process will inherently be controlled and steered by the majority, in any democratic country. It's the type of argument that usually comes from privileged white people who have never experienced what "due consideration" means. It's a completely ridiculous premise, because it does not fit reality. Not the reality of most minorities at least.
You keep saying this, and I am asking you: what were you prevented from saying? What Muslim voices in the West were silenced unfairly?
You insist it happened, but in response to what? What were you not allowed to say that was just so unfair. And no, being criticized for saying it doesn't count.
What were you prevented from saying "because it was hate speech" with no extra deliberation? It's almost like the content matters.