I also feel bad abandoning my wife and vice versa but I realized recently its still important to find space for that. After some mutual encouragement we've begun taking turns here and there watching the kids, purely for the other person to make plans and go out. Of course, it doesn't help making friends necessarily, but having that habbit does provide the space for it when the opportunities arise.