While I generally agree with you, it is interesting to note that the notwithstanding clause was specifically used to target free speech. And not just a little bit of speech, but any English on signage in Quebec (which, if I recall, must be written more prominently in French than in English).