The land would not be left unutilized or left to nature, and those same practices are used to produce all of your food. Take a drive down the central valley of California and the vast lettuce and fruit farms will have less genetic diversity than your backyard garden. Monoculture production is a consequence of efforts to increase production efficiency, regardless of the actual crop being produced.