My personal limit is 12-13 hours out of the house each day; more than that and it burns me out fast. It definitely wouldn't work for me with an employer who was less flexible about when the work day starts or who had expectations of large numbers of hours spent with butt in chair.
For me, I didn't really take advantage of the city - I like occasionally went to a museum or went out to eat but I mostly just sat on the couch watching TV and surfing the internet. So I moved north and got myself forty acres of tree-covered Bath-Nassau soil with rocky outcroppings. Now my free time is filled with rennovating a 200 year old vernacular house, turning my land into a proper hobby farm, and going to various kitschy festivals and hiking trails with my family. My commute is, frankly, the most relaxing part of my week, where I can sit quietly for 90 minutes and read a book or take a nap or post WoT conspiracy theories on Reddit.
It may not be for everyone, but it's amazing to me that I can have this lifestyle while also commuting to a regular old desk job in the largest city in the country. (It's also amazing to me that I'm part of a multi-century tradition of both commuting to the city from rural upstate and of gentleman farmers coming upstate from the city.)
Also don't forget Long Island, the Other In State.
(I'm one of the idiots who commutes ~2 hours from near Poughkeepsie, which is (a) still on commuter rail [the super-express Hudson line train takes ~90 minutes, plus the time to get to & from it] and (b) where half of the food you find in NYC farmer's markets comes from. I kinda like it.)