How is that any different from any other skilled labor? Recall the old joke about the plumber who charged $1 for banging on the pipe and $1000 for knowing where to bang. Even purely physical labor like digging ditches requires that you have certain physical traits (strength and stamina) that you can look at as assets.
> Society will not be worse off with most people doing physical labor.
Like what? There is virtually nothing that used to be done with human labor that can't be done by machine nowadays. Even ditch-digging is mostly done with backhoes, not men with shovels. For the moment we still need (some) humans to run the machines, but that's knowledge work too.
That can be true for physical labor as well. Fire fighters. Ambulance drivers. Soldiers. All get paid to be on-call.
Rent-seeking is different. Rent-seeking is the act of growing one's existing wealth by manipulating the social or political environment without creating new wealth [1]. That is not at all the same as making your living off of assets that you've earned with past labor.
What's the evidence that humanity flourished? The elites flourished off lots of slave and peasant labor. The standard of living was much lower while inequality was a lot higher. You would need to go back to hunter and gatherers, and it's hard to compare those lifestyles with today. For one things, billions of people don't live in nomadic tribes. Not everyone wants or can do physical labor. As a hunter gatherer, you had little choice but to pitch in.
The fact that you do not understand anything besides physical work as work is perplexing and, honestly, pathetic. It smacks of someone who has failed to make a life for themself, and is lashing out at anyone who is a viable scapegoat.
But there is some knowledge work where you are simply being paid in case people need you a small % of the time, and most of the time you’re not doing much or very trivial tasks. Like a consultant.
You secure an asset (specialized knowledge), and then you just live off of it while doing the minimal work you can, by turning yourself into an asset.
Even though this affords many people a comfortable life, much like being a landowner of a vast portfolio of properties, it contributes to the inequality and degradation of society, splitting people between those who must labor physically and those who look down at them comfortably from high balconies.
Society will not be worse off with most people doing physical labor. It was that way for thousands of years and humanity flourished. People may not like it, but humanity can only truly relax and do nothing when it has reached its peak and every problem is solved, and we are just not there yet. Back to work.