There is a sense in which being "productive" implies creating value, i.e., producing more value than you consume. If you restrict yourself to the realm of money, then being productive generally involves profit (e.g., producing something that gives net increase in monetary value over consumption costs). But if some person with leisure values things that other people generally don't want to pay money for (e.g., writing poetry), their leisure time can be productive (producing poetry) while they're purely an economic consumer (making no money, but spending money to live).
> There's a reason that retiring early is literally bad for your brain [0] - healthy adults are supposed to be engaged in productive activity. Whether productive activity means punching a clock, volunteering at a $nonprofit or striking out on your own, the point is that our minds and bodies need to be actively living, not just passively consuming
I would also agree that a person having some leisure that is "just passively consuming" is fine, maybe even good. But I'm pretty sure it's bad if _all_ of a person's leisure is "just passively consuming", at least if they have a decent amount of leisure (if you're working 80 hour weeks then it's probably not bad at all to have your leisure time all be passive consumption).
It's not really a hard-and-fast rule, but I suspect in many cases that leisurely-but-productive activities have output that is only nominally productive, like, I don't know, amateur painting, dime novels, learning a couple chords on guitar.
Really economically valuable output usually seems to require concerted work, which (even if fun) most don't think of as leisurely.