And entertainment, joy in the arts, categories of aesthetics have vastly improved as well. Look at art and film and music in the past five hundred years. Look at the works of Bach and anyone before him, he's clearly the greatest musician known in the world up to 1750, and since him in just 275 years, we've had hundreds of phenomenal musicians that outrank any before Bach. This is one mere slice of culture which is evidence of the vast improvement in human capability in modern times.
https://equitablegrowth.org/agriculture-the-worst-mistake-in...
https://www.discovermagazine.com/planet-earth/the-worst-mist...
Agriculture made us a lot unhealthier and shorter in stature (monoculture/unbalanced diets), and it also lead to massive power imbalances -- modern people spend more of their lives working.
This is a very mainstream observation -- it's also supported by Sapiens, the popular human history book of the last 5 years (recommended by Obama, Gates, etc.).
David Graeber's recent book argues that many early human societies purposely avoided agriculture, and then got "trapped" into the inequality of agriculture.
The argument is that there is enough for everybody, but agriculture leads to power imbalances, which make most people's lives miserable.
It's true that they were more miserable say in the 1500's in Europe than today. We have "recovered" from Europe of the 1500's, but that doesn't mean that this is the best time ever!