"You deplore the demonstrations taking place in Birmingham. But your statement, I am sorry to say, fails to express a similar concern for the conditions that brought about the demonstrations." From "Letter from a Birmingham Jail", which everyone should read: https://www.africa.upenn.edu/Articles_Gen/Letter_Birmingham....
This is a pretty uncharitable take, and one that I see too often.
I could just as easily say that people pressure others into turning every space political because they feel entitled to dominate every space with their politics, and they cannot tolerate any kind of truce or neutral ground. It's just another tactic to control the conversation.
And I think you're right! It seems very likely to me that that happens.
Playing devil's advocate for myself, however: the comment was not uncharitable because it did not attribute intent. Someone saying "let's keep politics out of it" may be trying to avoid conflict or simply be respectful, but unknowingly in effect quashing challenges to the status quo.
The comment was meant to be an observation of unwitting outcomes rather than secret intentions, but maybe that's in fact what you're objecting to ("it's easy to be pessimistic about the aggregate effects of small, inoccuous actions, and that attitude deserves some pushback").
Your comment has made me realize that I could have written my original comment more clearly; my dissatisfaction isn't about political agendas, but about deception – promoting free speech for the moment as a tactic rather than out of a dedication to the ideal, while claiming the latter.
Indeed, when people say "let's keep politics out of it", they're in effect trying to quell any challenge to the status quo, which arguably is inherently political, and this should probably be called out more often.