Entering postdoctoral for many fields basically means one is in a pre-run for a faculty job trying to be tenured, plus post docs need to do the real work rather than assign them to others. Unfortunately for many that is also the time a family or children begin to occupy private life, which provide extra stress and squeeze precious time. It is a highly stressed position, but not unique. If one wants to stay in academia the stress always accompany the entire career. I saw post doc sent baby to daycare at 6 weeks old, I heard post doc's promising job interview ruined by supervisor's letter. But they all survived and enjoy their research. I feel the freedom and joy of doing the thing interests you are also priceless and it is a rewarding experience even one is suffering.

It is highly stressed life, however, one can make it his own better. Keep positive, keep smiling.