While I very much endorse the growth theory (within reasonable limits I guess) and have experienced myself that things I didn't immediately find interesting grew on me, the downside for me nowadays is that I never really know whether things just aren't for me or if I'm just lazy and not trying hard enough.

Guess the trade-off of exploration vs. exploitation strikes even here, in career choices (or less consequential, hobbies and such).