>All I can think to myself is whether what I’m doing is really worth it compared to spending time with the baby.

Ever think that your baby may be thinking to themselves "Man, am I ever going to get a break from this guy?" also?