But I don't think it's true we weren't also eating a lot of animals, and in many cases, getting most of our calories from animals.
Separately, the existence of modern hunter-gatherer groups like the Hadza that eat a lot of meat and are healthy suggests eating meat is healthy.
(Of course, you have to be careful, since there are potentially large differences between the Hadza's nose-to-tail fresh, wild kills, and what you might get at the supermarket. Meat is a broad category.)
You also hint at traditional societies having much shorter lifespans than modern ones; it's true they have much higher rates of infant and childhood mortality, but lifespans for people who make it past childhood are very similar.
https://www.sapiens.org/biology/human-lifespan-history/
I agree evolutionary arguments in and of themselves will never be convincing (since there will always necessarily be a large amount of conjecture), but they are not being made in isolation: they providing an explanation for the observed fact that people living in modern societies are suffering a rising epidemic of degenerative diseases, and people living in traditional societies are not.
See also: the fantastic book "Nutrition and Physical Degeneration" by Weston Price.
There is a great deal of misinformation about hunter-gather diets: https://globalhealth.duke.edu/news/what-can-hunter-gatherers... or https://www.scientificamerican.com/article/why-paleo-diet-ha... (I didn't invest much effort in finding illustrative links, there are many more, and academic research).
Walking around constantly and digging up/harvesting edible plants, insects, fungi, possibly carrion, and the occasional meat (including organs) is much more likely.
Most of our ancestors' calories didn't come from hunting in most areas. Those that did had lifestyles so different from a modern one that it makes no sense to compare them.