Honestly, it just sounds like your job is too stressful. I work 5 days a week, sometimes more, but nobody bats an eye if I need to go somewhere 2 or 3 hours in the middle of the day. I much prefer this to 4 full time days. 5 days of less stress gives you more availability and more flexibility at the same time. I feel it's what we should really be striving for.