Then there are the second order effects, such as the dissolution of the middle class in your own country that everyone who outsources and automates is/will be partially responsible for.
Also culturally, we tend to share the wealth, so what is one job in America will become 6 jobs here.
From what I can see, outsourcing deprives one American of a job, makes one American desperate, and lifts 10 of us into the local middle class.
What can be zero-sum is if no new value is added, but you shuffle around who gets what. If Peter has a way to make $100 himself if someone else helps, and he decides to go from paying Mary $30 to paying Paul $5, he has enriched himself by taking from Mary.
I'm saying it's ethical for Peter to enrich himself as long as he enriches everyone, and it is unethical to take from others to give to himself.
The title of the book makes it sound like you can accomplish more by working less, while in fact the remaining 36 hours worth of work is done by someone else every week.
Think of it that way: How would someone from one of the developing countries the book mentions would benefit from what's written in that book?