I think what I'm suggesting is that if you have an idea that your passion might be gardening, you don't have to become a landscaper. You could work for a company that develops non-patented disease-resistant seeds, or a company that grows plants for sale in big box stores, or maybe even a small start-up that tries to connect small farmers that grow unique crops to restaurants that want to provide a unique menu.
Maybe it should be re-worded as "try to combine your skills with your interests".
I have a passion that I engage with for a living (math) and it's currently keeping me underpaid. I'm also very interested in art and so I go to museums, work on learning how to paint, etc -- and if it was my job, I'd stop liking it. I like it so much because it's a complement.
Some passions are like hot sauce -- a sprinkling makes almost every food better, while a full bowl gives you the runs.
This is 100x more grounded, clear-cut, and actionable than "follow your passion".
I'm glad you figured out what works for you, and I agree the mantra of 'follow your passion' is very misleading. But if we're all comfortable being where we're at nobody goes anywhere. And at the same time only following what you wanted also tends to lead nowhere.
As soon as I went back to doing what I am innately good at, I became much happier. I instead focused on improving my skills and having a positive impact at my workplace with the people I work with. This has lead to far more fulfillment than when I was "following my passion."
I also would like to add that I think "follow your passion" is largely broken because its hard to know what that even is a lot of the time. And then taking something fleeting and attempting to build a life on top of it is reckless. This week I might enjoy cooking and trying to get better at that. Should I then drop everything and go to culinary school? This week I enjoy carpentry, should I start a carpentry business? Now its gardening, should I become a landscaper? etc...