Not at all, I had only been in the industry for 2 years the first time I took 2 years off.. then did another 4 years at a desk and took many years off.

One of my secrets is being willing to relocate to really random and fun locations around the world - i.e. I moved to the Yukon where it was -40 for a few months every year. Every day was an adventure, and I got paid to live/work there.

Don't settle for good enough, go find something that enables the life you want to live.