I don't think the analogy sticks perfectly unfortunately.

I might still care about freedom of speech for the freedom of others which I value. I may not agree that I find value, assuming I have nothing to hide, in other people's privacy.

Not taking a position on privacy here, I'm just stating the analogy doesn't stick.