For produce you might look into local farms. Some have great subscription delivery services; some go the CSA route and require a little more work on your end.

I (in California) use farmfreshtoyou.com, which is subscription delivery from a local organic farm, and they're great. Your area should have similar offerings -- check out localharvest.org if you're interested.