You do realize that many places have these things called seasons? I do have a farm stand just down the road from me which has excellent (albeit a limited range of) produce in August. January? Not so much.
We're just not the agrarian society that would be conducive to providing "hyperlocal produce / butchers".
High quality food delivery is certainly possible. From what I've seen of Instacart delivering from Whole Foods, it works pretty well. But it's a premium service and isn't available where I live.
I'm not going to say it's necessarily more complicated of a problem -- I chose the word "complex" purposely -- as it mostly just simplifies down to being more training data for the AI produce picker (or what have you), but it sure seems like the failure rate would scale up rather quickly in real world scenarios. That would just lead to waste we wouldn't otherwise have.
Or maybe I just prefer to pick out my own produce. It could go either way, CV isn't my field.
Either way, handling pantry goods in this way doesn't sound particularly unreasonable.
One reason is because not all fresh goods (e.g. produce and meat) are created equal. Hand selection of those items is part of getting the most value our of your food dollars. You can likely overcome many of the other issues; price comparison, new product discovery, etc.; but you're still left with the inconsistency of items that come from dirt.