I think his definition of empathy is narrower than what many people think of when they think of empathy, so in some ways his argument struck me as a battle against a straw man. But TBH I haven't read the whole book (just coverage of it, and some of his related work).
There's a counterpoint here [2].
1: http://nymag.com/scienceofus/2016/12/paul-bloom-makes-a-weir...
2: https://www.psychologytoday.com/blog/good-thinking/201310/wh...
And yes, this applies solely to the situation I gave.
What is the depth that makes it sound good?
For example, someone gets caught with some crumbs of weed. You could make the empathetic argument for fining that person rather than jail time because, "They're not hurting anyone. It's just a little weed.", so on and so forth. Or you could make the logical/rational argument that the punishment and what stems from it (housing a person in jail, the economic aspects of ruining this person's life for what really is a victimless crime) does not fit the crime.
Seriously? At the very least I hope you're only narrowly applying this to the jury system, but even then...