Even this is a political decision, and is generally made based on which side the decider is on.
At least if you include the word genocide. And are also careful to not explicitly point out that Native American relocation and Black Slavery were both genocides that included lots of rape [2]. And are not in a school board meeting [1]. Etc.
[1] https://www.foxnews.com/media/author-sex-slavery-book-graphi...
These questions are political, even if you've made up your mind about them.
Is it the research community or the AI-selling industry? Because I see researchers and AI ethics advocates making that point (that toxicity is both subjective and more broadly contextual) all the time, and not just in a detached way but it is central to their arguments about openness and disclosure and public research on functionality; on the other side, I see censorship and adherence claims around particular political views of “safety” alongside opaqueness as the industry-leaders common response.
The flip-side is also true: a lot of people on the "anti-ethics" side of this debate are too coy about the fact that there are definitely parts of the definition of toxicity that the vast majority of people will agree upon.
And even if we could define toxicity, what to do about it isn't obvious. E.g., let's take something uncontroversial: graphic depictions of violent rape of children. Is this something we should suppress? Not necessarily. It depends on the context. E.g., in descriptions of war crimes and genocides, we shouldn't censor victims who want the audience to know what happened to them. But those same descriptions shouldn't be co-opted into erotica, for example.
One way "out" is to say something like: "look, what we're interested in is providing tools for enforcement of community norms when communicating with a given audience; the community/culture gives us its definition of toxicity and we provide the tools to prevent toxic generation". But those tools aren't neutral: that could be a description of guardrails for children, and could also be a description of overtly political censorship.
It's a difficult and fraught area, and I think all sides of the debate could benefit from more empathy for the other sides of the debate. In particular, this includes presumption of honest intent. There is nothing wrong with filtering toxic content per se -- we can almost all agree that it's reasonable if a company doesn't want to buy a customer service AI that sometimes quotes Mein Kampf to customers with Jewish last names. But the techniques for doing so are not context-free goods, either.