You don’t need to kill all plants, just enough of those we can eat. Not even all of those, either. Look into history to see what happens when enough people don’t have enough to eat.