Well, someone that finds speech offensive/upsetting/etc can pretty much call it hate speech. So in the end, hate speech can be anything that upsets someone.

I mean, you can reasonably define hate speech as something any reasonable person could agree is actually potentially harmful (like promoting some kind of prejudice in some dangerous ways), and I presume that's how it's defined under legal terms, but colloquially people most likely won't be doing that.