Often times people who work a lot are avoiding serious problems at home/family too, so even ordering someone to go home might be ordering them to go back to an abusive partner or something. I know more than one person who used “oh, my job takes so much of my time!” As a socially acceptable reason to avoid interacting with abusive people in their life.