I've struggled, and am struggling, with the same thing. The only thing that helps me is to keep doing... something.

Also, talk to your wife about changing lifestyles if you haven't already. She might surprise you.