The decline in human health post-agriculture is pretty well accepted within anthropology. Jared Diamond's discussion has been widely cited:
http://www3.gettysburg.edu/~dperry/Class%20Readings%20Scanne...
Cultures didn't flourish because they were in good health. They likely flourished because agriculture supported a lot of people within that culture, which let them dominate other cultures.
I have a great deal of knowledge in the domain of the English language. "Health" can mean a lot of different things. Support of the survival and dominance of a species is a great indicator that the diet is "healthy".
flourished because agriculture supported a lot of people
All you have to do is to refer to modern organizations like the WHO to know that supporting life for a lot of people is a great deal of the meaning of "health".
[edit: clarity]
In my mind, existing is healthy than not existing.
For example, height decreased post agriculture. Did that mean that agriculture didn't allow people to grow as tall, or did agriculture decrease selective pressure on being big?
Anthropology is not as rigorous as math, physics, chemistry, or even economics. Anyone talking condescendingly about things that are "well accepted within anthropology" from 10k or 100k years ago are way overselling the discipline.
That statement doesn't make sense.
It was because agriculture contributed so mightily to human health and well-being that we survived and flourished to cover the surface of the earth and travel into space.
Even if some of our ancestors were bigger, it doesn't mean that they were "more healthy".