There is research pointing to hunter gathers only needed to do a couple of hours a work daily, while neolithic people engaged in agriculture had to work much longer hours for the same kcal output.
https://www.thelancet.com/journals/lancet/article/PIIS0140-6...
This compares to a floor of 50-70 hours per week for premodern agriculture. The other side of it is that many of our estimates for agriculture come from hierarchical societies where a significant fraction of the work performed every year was heavily taxed. It’s hard to disentangle how much work was performed for sustenance of one’s family or village compared to the greater societies development (noble extravagance and all).
https://arheologia.com.ua/index.php/arheologia/article/view/...
I'm reminded of the "Alone" TV series where one contestant shot an 800lb Ox with a bow and arrow and single-handedly butchered it and carried it back 2 miles to his camp (requiring about a dozen trips).
Now if they can only all pull together as a team, it's double Whoppers for everybody. After they come out the winner :)
I also don't entirely follow your point that goes from 'backbreaking work' to 'sedentary lifestyle'. The latter only happened in the post war years when we worked out how to have all of the energy we need on tap. Before then, life was hard, albeit not for everyone since we have hierarchies.