Personally I think hate speech is a term which cold use more precision - I personally term it the rather uncatchy '<ethnic> intimidation' with ethnic substitute for the appropriate other term as it encompasses the parts of it that cause harm.

To use an example of an obscure half-joking religion just say calling Pastafarians a bunch of stupid useless colander-heads is hateful speech but not hate speech. Saying that we should all go out tomorrow and lynch Pastafarians would be religious intimidation and captures where the wrong is - not the speech but the actual 'verbal assault'.