I don't know about you, but if I said someone doesn't have empathy I would think that I'm the one lacking it, since I can't possibly imagine understanding that person or their feelings.