I'm not sure you understand what free speech is.

Being able to claim someone's speech makes you "unsafe" is akin to "words are violence" that the pro free speech crowd objects to. Elon may face a greater risk of harm with the world having a way to guess his location (broadly) but that's not a case to silence speech.

It should also be noted he's a billionaire. Have you seen Zuckerberg's security spending?