Your argument is already assuming its conclusion, i.e. that all speech should be free. But then you also recognize that, no, actually, some kinds shouldn't be (i.e direct threats).

The point is: that line you're drawing is arbitrary. You can define it narrowly and precisely, and I think that's what you tried to do here, but at the end of the day you chose where to draw it.