I am in a similar situation and it's usually just fine, but it becomes difficult on days when I would be better off sacrificing that "me time" to do something else (chores, errands, family obligations, etc.). When that happens, the only thing I can cut back on to meet those obligations is sleep.