Freedom of speech is too important to be of secondary value to something that can't even be defined clearly.
Edit for clarity: What you think is fine or not today may or may not be considered fine in the future. So there's an incentive to not set what is intolerance in stone on the basis that today's standards may not match those in the future.
Who defines hate speech? Mobs? Who gets to enforce the official definition of intolerant behavior? Trump? Hillary? (hyperbole to make a point) If we don't maintain freedom of speech above tolerance, then we cannot speak out when our words violate the accepted (and as you pointed out, transitory and unpredictable) definition of 'what is fine'.
Do you really want to just stop talking, without recourse, when a future mob decides your words are unacceptable?
https://en.wikipedia.org/wiki/Meese_Report
https://www.nytimes.com/1986/10/21/books/some-say-meese-repo...
Any hope of easy classification crumbles when you consider that a society can be too open (there are actual group that want to include age as a form of protected self expression, with significant consequences on the legality of various actions).
At this point is it being intolerant if you do not tolerate treating a 30 years old person as 8 years old in matters of law?
What I do know is that SCOTUS famously refused to define what porn is [0], and went instead with something to the effect of: I know it when I see it, and this is not it.
[0]: https://en.wikipedia.org/wiki/I_know_it_when_I_see_it
Methinks we as a society can arrive to similar types of rulings for intolerance.