I'm not in NYC (Phoenix here) so can't really comment on the local markets there. Here in Phoenix, there's a chain called Sprouts that has produce about as good as Whole Foods (or AJs) while being quite a bit less. As for meat there's Basha's (owns AJs) which is a bit less, similar quality. For everything else there's Fry's (Kroger) or Walmart. In the end, it depends on most of what I'm looking for as to what store I'll go to. In general it's only when I want more mushroom varieties fresh that I'll venture into an AJs or Whole Foods. I don't do the boxed "organic" anything, as it's usually just overpriced with dubious additional quality.

I do wish more meat/egg sources labelled what feed they use as I find that influences flavor far more than organic v. not.