I don't particularly agree with him, but Yale psychology professor Paul Bloom wrote a book called Against Empathy [1].

I think his definition of empathy is narrower than what many people think of when they think of empathy, so in some ways his argument struck me as a battle against a straw man. But TBH I haven't read the whole book (just coverage of it, and some of his related work).

There's a counterpoint here [2].

1: http://nymag.com/scienceofus/2016/12/paul-bloom-makes-a-weir...

2: https://www.psychologytoday.com/blog/good-thinking/201310/wh...