We all should be making sure we're hearing stuff properly. I don't think many of these things are _really_ about misunderstanding, but it has happened.
Inversely, just because the speaker says "I meant X, but I didn't mean for it to offend" doesn't completely absolve the person. The words have meanings, and if you want to share you ideas, it's important to do it properly (just like you should know how to drive before going out there with your car).
In terms of speech that offends, we can only control what we say, not what people think about it, nor whether their thought is reasonable. Therefore, punishing someone for what they say because it offended someone seems pretty illogical, unless you can make a damned good case, based on context, that it was probably going to offend.
A simple way to resolve that question is to ask the person who said it, "Why did you say that?" If they have a response that is not "because I wanted to offend" that is also reasonable, then I'd suggest the onus falls at least equally on the person who claims offense.