You're confusing two concepts that share a word: Your identity in the sense of self worth and personal ideals, and other people's view of you, your identity to them - Their interpretation of the former, to some extent, but also their judgements on your trustworthiness.

It's the latter that's being stolen. It's a crime against both you and your friends and creditors.