I think you have a misconception here about what freedom is. A freedom is a restriction on the state. It simply says that the state cannot act against you for exercising a freedom, or act to prevent you from doing so. Repercussions, consequences, or retaliations are completely unrelated so long as they are not state-sponsored.

With this in mind, "I do not agree with what you say but I will defend to the death your right to say it", and "freedom of speech doesn't mean freedom from consequences", are totally compatible. You can both fight for a restriction on state power, and have a personal reaction to what somebody has said.