That's probably true. However, like I said above, our agriculture is also unsustainable. Most of the increase in crop yields over the past few decades have come as a result of fossil fuel-based fertilizers. Without oil yields would plummet. Overpopulation is the problem.
Ideally, animals should be raised either on land that's not suitable for agriculture, or on the farm, eating the inedible plant parts and providing more of a closed-loop ecosystem.
Of course if people wanted to eat much less meat, and if there were a lot fewer people, things might work better sure.