As another person said, its more about being "out of sight, out of mind". But the biggest component is cost and availability. Large scale farming (unsustainable over the long term) with a single crop per season and minimal rotation allowed for huge crop yields over the short term which drove prices down. At this point the majority of consumers do not want to pay more than they've become accustomed to paying for any fruits or vegetables.

Sustainable farming costs more in the short term. Over the long term it is the only reasonable way to farm.

The book "Lentil Underground" by Liz Carlisle was a great read on this topic.