>my freedom to not listen ends where I prevent you from speaking.
But that is exactly what is happening. Liberals, or libtards would be the more correct term, are drowning everything they think they disagree with, creating echo chambers. This is what divided our societies and brought us Trump, Farage, LePen, etc.
I will defend people's right to say things I don't like hearing. However I will also defend other people's right to not listen (as long as they don't try to prevent the speech from happening) - that is also a freedom.
To modify a quote: Your liberty to speak ends where I am forced to listen to what you are saying, my freedom to not listen ends where I prevent you from speaking.
Downvoting does not prevent the speech from happening. It merely says I don't like this. Flagging does...