Apply huge amounts of nitrogen fertilizer to grow corn, badly contaminating the Mississippi river and creating a dead spot in the Gulf of Mexico, feeding animals in CAFO where discarding of the manure is a problem and that's something different.
Real life systems are a little more complex than that as a cow might be grown up on scrub land and then fattened up at a CAFO for the last few months. Who knows what the long term fate of land that is cleared in the Amazon rain forest is.
Most land is scrub. You can like leaving more of it that way, but no one else will notice in percentage terms.
The type of food you get from animal agriculture is nowhere near the same thing as what you get from large scale plant agriculture.
On top of that, what you harvest from plant agriculture needs enormous amounts of processing and requires a whole lot of modern machinery and a very complex supply chain. And at the end of it the nutritional profile of your output is not at all the same, you can get closer by adding even more processing, that requires even more supply chain wizardry, more energy and more secondary input but suddenly you are not really more efficient than the animal farm; how shocking...