What fantasy world is everyone living in that they love their jobs and derive pleasure from it? What 1%'s of gardeners are we talking about here? I assure you 99% of them would rather not be gardening.
It's very likely if you are doing a physically laborious job you are making very little money, struggling to feed your family, give your kids the spoils they want, your wife is mad that you can't take her on vacation etc. etc.
Granted, it isn't that physically demanding. The biggest amount of labour I find is repairing/modifying equipment, and that comes with interesting problem solving challenges, which my programmer mindset tends to enjoy.
Still, I can see why gardening would be seen as appealing.
> your wife is mad that you can't take her on vacation
Why would anyone get mad about that? My wife is a grown adult, she can pay for her own vacation. Farming is my vacation.
All I can say is that you are incredibly lucky if this is a true statement.
Do you know people who do that kind of work? I spent three years working with a decent sampling of them. Sure, some feel that it's drudge work and that they're stuck doing it for lack of options. They get through their day doing the minimum necessary - you can find those types in our field as well. But a lot do take real pride in what they do, and in doing it well.
Of course many if not most would take an office job for more money if they opportunity were there - but that doesn't mean none of them derive pleasure and satisfaction form the work they're doing.