Indeed, if you add up work + leisure, and subtract out 8 hours a day of sleep, there's 32 hours a week left over for men, and 38 left over for women. I wouldn't be surprised if a lot of this time precludes leisure activities because of the "on call" nature of being the primary care-giver.
Moreover, simply adding up housework + childcare time doesn't quite capture the situation. Spending a day taking care of a kid is much more exhausting than working at a typical white-collar job. It's blue-collar work that cuts across the economic classes in a world where most men aren't doing blue-collar work. Last year I had a 9-5, while my wife had a busier schedule. So I'd cook and watch our daughter until she got home. At 8:00 when my wife got home, she had much more energy for leisure activity. I'm a guy who regularly works 70-80 hour weeks, but a day taking care of our toddler puts me right out.