The mapping between natural features and human population/agriculture is generally near a local optimum. If a random grassland would be swapped with a random desert of the same size, from a natural viewpoint it would be nearly the same, but it would have horrible consequences for the people living there.

Any significant changes to the natural features in random direction should be expected to be harmful for us - we can be rather sure that moving 10 steps in direction A is expected to be worse than moving 1 step in direction B even if we don't know anything about the actual changes caused by those directions.

We should prefer small random changes to big changes, unless we're really, really sure that the big changes are actually beneficial.