Not sure about Canada, but in the US, this mostly due to deforestation and centuries of aggressively farming the same land: there’s preciously little nutrients and carbon in the soil compared to fields that have living forests right around them.

Sure there would be crop losses to animals in such fields, but those animals also fertilize the soil and keep the microbiome diverse.