I see it completely the opposite. If someone has already made all the millions they'll ever need, it's better for society to free up that job for someone else who needs the income instead of keeping amassing more and more millions to one single person. Paying it forward here is opening up the opportunity for the next person.
A life of "leisure" still contributes to society. I would of course still pay bills and buy things, supporting business. I'd still pay my taxes, supporting the government. I'd still volunteer, like I do now outside my job. I'd like to teach, write, and create more (I like to sing, make music, art, furniture, etc). And I'd really like to start an informal class for underprivileged youth to learn computer skills and get a job without going to college, like I did.
If I didn't have to work, I'd have a hell of a lot more time to make the world a better place. In addition to the selfish kayaking, cooking, traveling, etc, that I already do with a job.
Rather, I would imagine the "not-selfish" version of this would be to spend your life doing those things, and finding ways to help others get to where you're lucky to be.
I just find the "I'll get mine and fuck off" attitude to be less than ideal.