I would sum it up as in "your job is not the most important thing in your life". When you're juggling with your 5 balls and you reach a point where you can't handle it anymore and need to drop a ball, dropping the work part is not as big of a deal as the rest. E.g. if you do something just to keep your job but that hurts other people or your health, don't do it, it's not worth it: don't stay late every night, when it's forbidding you to kiss your kids to bed; don't accept to be dropped off in a war zone just to keep your job, etc.
The opposite hypothesis seems pretty absurd. You wouldn't argue that, e.g., if you have to disappoint your boss or your spouse, your boss is more likely to remain unconditionally faithful to you.
People don't exploit this, and that's a good thing. But if I were trying to avoid harm in both my professional and personal life, I'd know that personal connections are more loyal, and thus I can more afford to let them suffer in the short term.
I guess health is an exception there. I'm not sure what "spirit" means, but it seems hard to change.
You'd have to be a pretty shitty human being to exploit this, but it is true. The transactional nature of business means it's the least amenable to long-term relationships, and requires the most aggressive maintenance if you want to optimize it. Most people naturally behave in a way that counteracts this: they could abuse the trust of their loved ones, but they don't.
So perhaps he should have said something like "business is a glass ball. The others are all hamster-balls, with little hamsters in them. If you drop the hamster-ball, it's not going to kill the hamster; it might not even seriously hurt it. But don't be an asshole; if you're going to drop something, drop something that won't be hurt."