"nutritional standards of Neolithic populations were generally inferior to that of hunter-gatherers, and life expectancy may in fact have been shorter, in part due to diseases. Average height, for example, went down from 5' 10" (178 cm) for men and 5' 6" (168 cm) for women to 5' 3" (165 cm) and 5' 1" (155 cm), respectively, and it took until the twentieth century for average human height to come back to the pre-Neolithic Revolution levels."
The citation is to a Michael Shermer book.
[1] http://en.wikipedia.org/wiki/Neolithic_Revolution#Consequenc...
http://www.ditext.com/diamond/mistake.html
Edit: As others have mentioned below, the main evidence is a decline in the height, bone density and tooth health of skeletons of pre and post neolitic humans.
Average lifespan also declined, from 33 --> 20 according to Wikipedia. (I don't know if the studies cited are good sources)
Keep in mind that these lifespan figures include infant mortality. This explains why they remain so low up until very modern times.
http://en.wikipedia.org/wiki/Life_expectancy#Life_expectancy...
http://en.wikipedia.org/wiki/Mark_Nathan_Cohen
EDIT: He's quoted in the article you linked, actually.
It seems that some of the cited stuff is still too inconclusive to fully draw the conclusion farming -> decreased "body fitness", but there definitely seems to be some tendency there.
I am not an anthropologist.
Much can be done with forensic anthropology. There is one North American area in present day Mississippi that was inhabited across the adoption of agriculture, and the decline in health is documented in the teeth and bones.
Do you have any proof for this statement:
>there was a clear dip in human health once we entered the neolithic era
This statement seems hard to prove seeing as we didn't keep any health-records between 10,000 BC and 2,000 BC (I can see, however, that we currently live quite unhealthy).