If you trust the person to not be acting on malice, then I'd not worry too much about the precise semantics of those words and guess that the person is mainly expressing their frustration, hurt, etc.

What about responding to that honest emotional expression directly? "Ouch. I hadn't realize how much I'd hurt you. Truly, I'm sorry."